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ABSTRACT 

In 1990, the National Education Goals Panel— six 
governors, four members of the President's administration, and four 
members of congress — enlisted the aid of six resource groups (RGs) of 
educators, business psople, and technical experts in identifying: 
which indicators best measure progress toward six national education 
goals, data for the panel's first annual progress report in September 
1991, and the content of future progress reports. Each RG included 
between 8 and 10 members. The six goals state that by the year 2000: 
(1) all children in the United States will start school ready to 
learn? (2) the high school graduation rate will increase to at least 
90%? (3) U.S. students will leave grades 4, 8, and 12 having shown 
competency in challenging subject matter including English, 
mathematics, science, history, and geography, and every U.S. school 
will ensure that all students learn to use their minds well, so they 
may be prepared for responsible citizenship, further learning, and 
productive employment in the modern economy? (4) students in the 
United states will be first in the world in science and mathematics 
achievement? (5) very U.S. adult will be literate and will have the 
knowledge and skills necessary to compete in a global economy and 
exercise the rights and responsibilities of citizenship? and (6) 
every U.S. school will be free of drugs and violence and will offer a 
disciplined environment conducive to learning. This document presents 
six RG interim reports corresponding to the six goals. The reports 
highlight ways of measuring and monitoring the nation's/states' 
progress toward meeting these goals. Lists of the membership of each 
RG and instructions and a form for submitting public testimony for 
panel consideration are included. (RLC) 
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INTRODUCTION 

In 1990 the President and Governors of the United States agreed upon the six National 
Education Goals. Their purpose was to help improve the quality of education by setting 
high standards and focusing attention on how well our society is able to achieve them. 
The National Education Goals Panel, composed of six Governors, four members of the 
President's Administration, and four members of the United States Congress, was 
established to report on the Nation's and States' progress towards meeting these goals. 

Resource Croup Reports 

The Panel asked six Resource Groups of nationally recognized educators, business people 
and technical experts to help them identify what indicators would best measure progress 
towards each of the six goals. These Resource Groups were assigned two principal tasks: 

• To identify what data are available to report upon in the first annual Progress 
Report (September 1991); and 

• To suggest a vision, unconstrained by the limitations of current data, of what would 
be desirable and needed for Progress Reports in the future. 

The Resource Groups met from January through March of 1991 to discuss these issues. 
Their ideas were transmitted to the Panel at its meeting on March 25. This Compendium 
of Resource Group Interim Reports is the initial product of the Resource Groups' 
deliberations. A companion report, the Discussion Document, includes abstracts of the full 
reports and is available upon request. Please address your inquiries to: 

National Education Goals Panel 
1850 M Street NW, Suite 270 
Washington, DC, 20036 
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Chapter 1 
Readiness for School 

An Interim Report From the Resource Group on School Readiness 



GOAL 1: By the year 2000, all children in America will start school ready to learn. 

Objectives: 

• All disadvantaged and disabled children will have access to high quality and 
developmentally appropriate preschool programs that help prepare children for 
school. 

• Every parent in America will be a child's first teacher and devote time each day 
helping his or her preschool child learn; parents will have access to the training and 
support they need. 

• Children will receive the nutrition and health care needed to arrive at school with 
healthy minds and bodies, and the number of low birthweight babies will be 
significantly reduced through enhanced prenatal health systems. 
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In early 1991, a Resource Group on School Readiness was convened by the Panel to 
recommend indicators and strategies for measuring progress toward achieving this goal. 
Members of the group arc as follows: 



Ernest L. Boyer 


The Carnegie Foundation for the Advancement of Teaching, 




Princeton, New Jersey (convener) 


James P. Comer 


Yale University, New Haven, Connecticut 


Donna Foglia 


Evergreen School District, San Jose, California 


Sharon Lynn Kagan 


Yale University, New Haven, Connecticut 


Samuel Meisels 


University of Michigan, Ann Arbor, Michigan 


Lucilc F. Newman 


Brown University, Providence, Rhode Island 


Doug Powell 


Purdue University, Lafayette, Indiana 


James Wilsford 


Orangeburg School District, Orangeburg, South Carolina 


Nick Zill 


Child Trends, Inc., Washington, D.C. 



The Panel and the Goal 1 Resource Group welcome your reactions to the Interim Report on 
School Readiness. 
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National Education Goals 
Goal One 

Readiness for School 



I. The Charge 

In his first State of the Union message, President Bush announced six goals for all the 
Nation's schools, which then were endorsed by Governors from all fifty States. Subsequently, 
the National Governors' Association (NGA), working closely with the White House staff, 
created a National Education Goals Panel to monitor national and State progress in meeting 
these new goals. Our assignment, as a resource group to the Panel, was to focus on the first 
goal, school readiness. Specifically, we were asked to suggest ways to measure progress 
toward reaching the first goal, propose new assessment tools if needed, and tell the Panel 
what strategies we do not recommend. 



II. Convictions and Cautions 

We begin with the conviction that having children come to school ready to learn is the 
Nation's most essential education goal. \ good beginning for every child is the key to all 
other goals, and if a solid foundation can be laid in the first years of life, prospects for school 
success will be dramatically enhanced. Further, there is disturbing evidence that, today, far 
too many children come to school strikingly disadvantaged, with great handicaps to learning. 
Giving priority to early childhood development is, we believe, an essential clement in 
preventing such barriers to education. 

The concept of "school readiness," while critically important, is difficult to define, and 
even more difficult to measure. Current assessment tools are very crude, often measuring that 
which matters least. Further, young children are developmentally not ready to perform 
SKillfully on paper-and-pencil tests. There is a grave danger that, in our eagerness to 
quantify the learning process, we will put numbers on little children and make inappropriate 
and destructive judgments about who they are and what they might become. 

We also underscore the point that young children develop at different rates. It is 
extremely risky to gather information at a fixed point in time and then generalize about a 
child's "readiness." Further, we are deeply concerned about the increased inclination in our 
culture to rob children of their childhood, to impose academic expectations far too early, and 
to ask children to conform to predetermined standards. In short, the assessment of young 
children raises issues that are critically important, both educationally and ethically. Such 
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efforts, however well intended, must be carefully directed and, above all, should avoid doing 
more harm than good. 

With these concerns very much in mind, we fully agree with the panelists who said at 
the February meeting of the Goals Panel that a national preschool readiness test, a kind of 
prekindergarten SAT, should not be used. We also conclude that school entry should not be 
linked to the evaluation of the readiness of young children. And, further, we reject the idea 
of using assessment measures prematurely to label or track individual students. 

Still, caveats notwithstanding, we are convinced that this is an historic moment for 
American education. National goals are, we believe, critically important, and appropriate 
ways must be found to measure results. Now is the time to think carefully about school 
readiness and propose creative new ways to expand, rather than restrict, the opportunities of 
children. The objectives of such efforts should be to raise public awareness, monitor State 
and national progress, and guide positive action to improve educational prospects for all 
children. 



III. Defining "Ready to Learn" 

As an advisory committee, our first task was to consider the essential question, "What does it 
mean when we say that 'all children should come to school ready to learn?'" Strictly 
speaking, every child, except the most severely impaired, is ready to learn from the first 
breath of life. Further, children come to school "ready or not," and many educators insist, 
quite appropriately, we believe, that classrooms must adjust to meet all needs. Viewed from 
this perspective, one could argue that the right question is, "Are schools ready for the 
children?" Still, the handicaps many children now bring to school should not be casually 
accepted. Our goals should be to have every child bom and reared so that his or her 
capacities for learning will be enhanced, not diminished. Guided by this objective, we agreed 
that, for the purposes of our assignment, being "ready to leam" means being prepared to 
participate successfully in formal schooling. 

IV. The Dimensions of Readiness 

Second, we examined the various dimensions of school readiness and found, not surprisingly, 
no agreed-upon definition. Many professionals insist that the focus should be primarily on 
verbal skills; others emphasize general knowledge, while still others look at the child's 
"developmental maturity" for school. We conclude that readiness docs indeed involve the 
whoie child. It relates not just to verbal proficiency, but also to emotional maturity, social 
skills, attention span, and, at the most fundamental level, the child's physical condition. 
Specifically, we propose a view of school readiness that embraces the following five 
dimensions: 

4 
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• First, Physical well-being. A large and expanding body of research clearly links 
maternal and child health to school performance. It is clear that such conditions as 
low birthweight and poor nutrition may, in fact, restrict a child's capacity to learn. 
We recognize, of course, that some health problems may not have lasting effects on 
a child's preparedness for school. Still, a child's health is a basic dimension of 
school readiness, one that must be carefully assessed. 

• Second, Emotional maturity. Bending is a basic human need, and to enter school 
ready to learn, a child should feel self-confident and secure. He or she needs to be 
loved and supported by adults in an environment that is stable and secure. A child 
who is denied emotional support in the first years of life will very likely come to 
school less well prepared to learn. 

• Third, Social confidence. School readiness also means being able to interact 
successfully with other children and adults. In the classroom, young students must 
be able to live with group constraints that include taking turns, following directions, 
and completing other tasks that make possible a creative and productive classroom 
climate. 

• Fourth, language richness. Proficiency in language is at the heart of learning, and 
children are aided enormously if they arrive at school linguistically, or, as some 
might put it, "cognitively," empowered. A child who has been involved with 
books, who has engaged in conversation with adults, and who has learned to value 
language is advantaged. School readiness is enormously enhanced for such 
children. 

• Fifth, General knowledge . To live is to learn, and children whose environment has 
been filled with rich experiences during the preschool years bring great capacity to 
the classroom. The more young children are exposed to a wide range of activities 
and creative play, the more knowledge and information they acquire. A strong 
foundation for further learning is put in place when a child has been encouraged to 
learn from his or her surroundings. 

Clearly, readiness should not be narrowly defined. Rather, it should be viewed as a 
pattern of qualities, a cluster of conditions and characteristics that, taken together, enable 
children to take full advantage of the opportunities and demands of formal schooling. We 
insist that any effort to assess the school readiness of children involve the five essential 
principles summarized above. 
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V. Assessing Readiness: What Are the Tools? 

This brings us to the crux of the assignment. Is it, in fact, possible to evaluate the school 
readiness of children? If so, what yardsticks are available now, and what instruments need to 
be developed? 

We conclude that the readiness of children can, in fact, be evaluated. And we also 
believe that both indirect and direct measures should be used. By indirect measures we mean 
data that are not derived directly from the child, but that still provide important insights about 
experiences and influences that affect the child's potential for learning. We are confident that 
information about children's physical and environmental experiences before they come to 
school is related directly, and critically, to readiness. 

But such information — sometimes called "proxy data" — while valuable, is not 
sufficient. It can shed light on the degree to which certain important preschool conditions 
have been met. However, indirect data cannot tell us the extent to which children do, in fact, 
have the knowledge, skills, and attitudes needed for formal learning. Therefore, we also 
propose that additional information be gathered directly from children to reveal more 
precisely how well they have been prepared for school. 

In order to gather both direct and indirect data, we recommend monitoring children's 
readiness at three points: before school, at school entrance, and in school. In the following 
section, we outline this three-step plan, along with a timetable that spells out what data are 
available for fall of 1991, and the steps that should be taken to move toward a full report by 
the year 2000, 



A. Before School Assessment 

Readiness to learn necessarily begins long before school, even before birth itself, and we 
conclude that "school readiness" relates to the child's health, to the home environment, and to 
preschool education. Indeed, the President's and Governors' three "ready-to-lcarn" objectives 
focus directly on preschool issues and provide, we believe, an excellent framework for 
identifying measures that might be used to indicate whether young children are being 
prepared for formal schooling. 

I. Health and Nutrition Factors 

Children who arc undernourished or who arc in poor health are educationally at risk. 
This fact was recognized by the President and the Governors, who declared: "Children 
will receive the nutrition and health care needed to arrive at school v/ith healthy minds 
and bodies, and the number of low birthweight babies will be significantly reduced 
through enhanced prenatal health systems." 
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There are, we believe, reliable information sources that can be used to monitor 
progress toward meeting this objective. And we recommend that the following data 
bases be considered; 

■ Birthweight and prenatal care . Vital statistics data collected annually from the 
States include information from birth certificates. For our purposes, we recommend 
that two items from this source be used: birthweight and the timing of prenatal 
care. 

* Nutritional status of children . The Nationwide Food Consumption Survey, 
Continuing Survey of Food Intakes by Individuals, provides the best data on 
children's nutritional status. In this annual survey, conducted by the U.S. 
Department of Agriculture, mothers report on their children's diets. We 
recommend that such data be included in the national assessment of readiness. We 
also suggest that the Department of Agriculture add questions related to hunger to 
its survey. If reports on subgroups are desired, it may be necessary for the survey 
to "oversample" young children. In any event, the current sampling design will 
have to be changed (not simply enlarged) to yield State-by-State data. 

• Children's access to health care . The National Health Interview survey, conducted 
annually, provides national (but not State-by-State) data patterns about family 
health care. Using this instrument, households with young children can be 
identified ("children ages 0-4" is their current classification, but a different cut — 
say, "children ages 0-6" — could be requested for purposes of the national 
assessment of readiness). The sampling design, however, must be altered if State- 
by-State data are desired. 

The National Health and Nutrition Examination Survey (HANES) involves an 
actual health examination of those in the sample. As currently administered, it 
could not be used annually or State-by-State. Data from HANES III will be 
"available in 1992, and in 1995, while HANES IV is being planned to provide data 
for the year 2000. In years when HANES is conducted, results could be included.. 

2. Home and Parenting Conditions 

Parents are, or should be, the first and most important teachers, and it is in the home 
that school readiness is most essentially achieved. These factors were recognized by 
the President and the Governors when they stated: "Every parent in America will be a 
child's first teacher and devote time each day helping his or her preschool child learn: 
parents will have access to the training and support they need." 

Here again, there are existing sources that provide data regarding home and 
parenting conditions. The following points should be considered: 
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* Parental status . Research indicates that school performance relates to the age and 
education of the mother. Therefore, we recommend that vital statistics — available 
annually on a national and State-by-Statc basis — be used to locate information 
regarding the age and educational status of the mother. 

■ Home activities . The National Household Education Survey (NHES) is a promising 
new instrument that we recommend be used to provide information regarding 
attitudes and activities in the home. Developed by the Department of Education, 
this survey asks parents about such activities as reading, television viewing, 
museum visiting, games or sports, and arts and crafts activities. It also asks about 
child care and preschool programs. According to the present schedule, national 
data will be gathered in 1991 and 1993, and then annually. 

We recommend that this important survey be expanded to include other 
parenting questions. We also recommend that the sample be expanded, annually or 
at least periodically, to collect data on a Statc-by-State basis. 

Finally, we recommend that the National Household Education Survey be 
supplemented periodically through a national suhsample for home visits using an 
instrument like the HOME scale developed by Dr. Bettye Caldwell. This screening 
procedure, used by home observers, has eight components and is useful for 
studying the developmental environments of 3- to 6-year-olds. 

* Parent education programs . As the President and Governors acknowledged, parent 
education is significantly related to the school readiness of children. Therefore we 
urge that the adult education section of the National Home Education Survey be 
amended to ask specifically about parent education, both formal and informal. 
Such questions could cite not only adult programs, but also classes parents took 
when they were students. The focus would be not only on participation, but also 
on parents' perception of the quality and value of such programs. 

3. Preschool Programs 

It is well established that access to high-quality preschool programs is of crucial 
importance in preparing children for school, especially those who are most at risk. We 
strongly endorse the objective that states, "All disadvantaged and disabled children will 
have access to high-quality and developmentally appropriate preschool programs that 
help prepare children for school." 

In order to assess national and State progress toward meeting this objective, the 
following data sources are proposed: 

Participation in preschool programs . The Census Bureau's Current Population 
Survey has an annual School Enrollment Supplement that provides national data on 

8 
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the participation of low-income children in preschool. State counts are available 
for children participating in special education programs under the Education for All 
Handicapped Children Act. 

In order to interpret these data, we also necc' to know how many 
disadvantaged and disabled children there are in the population as a whole. 
Fortunately, this information can be derived from the same two sources. 
Specifically, the current Population Survey can tell us, nationally and on an annual 
basis, the number of young children living in households with incomes under the 
federal poverty line. Statc-by-State data from the current Population Survey could 
be commissioned periodically. 

We should note that definitions of "disabled" vary in early childhood, and no 
complete census of young disabled children exists. However, an estimate can be 
derived, if the proportion of disabled among preschool-age children is assumed to 
be similar to the proportion of disabled among older children (as indicated by State 
counts of children getting services for disabilities in elementary school). 

• Quality of preschnol programs. The information now available provides only a 
small window into the interior of the preschool world. The best, but still limited, 
source is the Program Information Report Questionnaire sent out annually to all 
Head Start programs. We recommend that minor alterations be made so that this 
report can provide information about the quality of Head Start programs. 

A more comprehensive program of data collection on the quality and 
developmental appropriateness of preschool programs will be needed for the future. 
We specifically urge the use of recommendations developed by the National 
Association for the Education of Young Children. 



B. School Entrance Assessment 

Starting school is— or should be— a significant rite of passage for all children and their 
families. The events surrounding school entry should, we believe, be more celcbrativc. 
Further, the information schools receive about incoming students should be more complete 
and uniform across the country, since so many children move from one district to another. 
Therefore, we conclude that school entry, which typically means starting kindergarten, offers 
a critically important opportunity for the collection of data, one that could greatly expand our 
"baseline" of knowledge regarding the school readiness of children. 
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1. A National School Entry Form 

Parents are a valuable source of information regarding the readiness of children, 
especially at the time /hen school begins. But as things now stand, this opportunity is 
largely overlooked. Most schools secure little or no information from parents about 
their children, except perhaps birthdatc, address, and vaccination status. 

Therefore, we recommend that by 1993 a National School Entry Form be in place 
as part of the national assessment of school readiness. This new entry form would ask 
parents or guardians to provide information about their child's birth, health, language or 
languages, household and family life, as well as about their child's daycare or preschool 
experience. We also recommend that the entry form welcome parents to the school, 
remind them of their new responsibilities now that their children are in kindergarten, 
and advise them about where they can seek training or support. 

As part of the national and State-by-State assessment, a sample of responses 
would be sufficient. But schools and districts might also wish to use such a form and 
tailor it to their own special circumstances. Still, a common core of data would be 
invaluable in monitoring school readiness. Further, with a uniform format the proposed 
national school entry form would be transferable from one district to another, and 
become part of the student's permanent school record. 

2. A National Health Screening Form 

We have made the point on several occasions that prenatal care and good nutrition are 
closely linked to school readiness. Therefore, in addition to following these national 
trendlines, we conclude that health data should be gathered from children at the time 
they enroll in school. Specifically, a national health screening, performed by the school 
nurse or by a physician's assistant, could provide information about vision, hearing, 
immunizations, and general physical health, and identify, of course, children with 
special disabilities. Samples of health screening reports could be selected for national 
and State-by-State assessment purposes. But if universal, such screening would enable 
all children with health problems to be referred jt further examination. 



C. In-School Assessment 

This brings us to the third point of readiness assessment. In order to obtain information 
directly from the children, we recommend a national sampling of kindergarten students every 
year. We are encouraged to find that creative research is now going oti that does provide 
information about children that can be used in constructive ways to improve preschool 
experience and enhance learning. 
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Here, we underscore two essentia! points: 

■ First, such information is not to restrict school entry, nor is it to track individual 
students. If local schools wish to use such data for instructional purposes and 
teacher guidance, that would be at their discretion. 

• Second, the framework used to assess kindergarten children should be based on the 
five dimensions of school readiness defined above: physical well-being, emotional 
maturity, social confidence, language richness, and general knowledge. 

With these caveats in mind, we propose an in-schooi assessment process with four 
interlocking parts. The first involves gathering information directly from the child; the 
second involves gathering information from parents; the third calls for systematic teacher 
observations; and the fourth uses a portfolio of the child's work. The proposed assessment 
would occur at different intervals during the kindergarten year, and teachers would be vitally 
involved throughout. 

/. ChUd Development Profile 

The first step is to secure responses directly from children — but not by using a group- 
administered test. Rather, the proposed child development profile, given early in the 
kindergarten year, would use an individually administered instrument, one that could be 
scored to yield a profile based on the various dimensions of readiness and provide 
quantifiable data for a national profile, as well as State-by~State comparisons. We 
caution that it is absolutely essential that valid and reliable measures be used. And 
while several well-regarded instruments are currently available, more development in 
this area will be required. 

2. Parent Report 

The second part of the proposed in-school profile would involve gathering additional 
information from parents. Such data would go beyond the school entry form, asking 
selected parents to describe the child's readiness in each of the five dimensions. We 
remain convinced that parents are a vital source of information, and that they should 
continue to be involved in the assessment process, even after the child has begun 
formal education. 

3. Teacher Observations 

Third, we propose that teacher observations be gathered during the second half of the 
kindergarten year. Specifically, teachers would record in narrative fashion their 



11 



Resource Group Interim Reports 



observations of student performance— again following the five dimensions of school 
readiness. This process goes on all the time in a good classroom. However, we 
propose that it be made more systematic and more uniform, and codified so that the 
readiness of children can be better understood. 

4. Performance Portfolios 

Fourth, we would include in a proposed readiness profile samples of student work 
gathered during the second half of the kindergarten year. The portfolio could include 
drawings and paintings, stories dictated to the teacher, small construction projects, tapes 
of oral presentations, field trip reports, photographs, and a list from the teacher or 
parent of the picture books the child has enjoyed. Unlike the blurred "snapshot" of a 
standardized test, the child's performance portfolio would allow a more qualitative view 
of school readiness. 



VI. A Proposed Calendar 

What we propose, then, is a comprehensive assessment grid of school readiness — one 
extending from prenatal care to the first year of school. And we strongly urge that both 
indirect and direct data be gathered at three points in time: before school, at the time of 
school entry, and during school itself. 

We recognize that only a few sources of data are available today. This is not 
surprising, since the commitment to national goals, and the monitoring of those goals, is just 
emerging. Clearly, time will be needed to create new instruments and enlarge existing data 
bases in order to have a full profile of national and State-by-Statc performance. The 
following table summarizes the full range of our recommendations and the timetable we 
propose. 
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SCHOOL READINESS ASSESSMENT 



1991 1995 2000 

National S^tc National State National State 



Before-School Data 

Maternal/Child Health Data 

Vital Statistics X X X X X X 

National Health X 0 X 0 X (X) 

Interview Survey 

Nationwide Food X 0 X 0 X (X) 

Consumption Survey 

National Health and 0 0X0X0 

Nutrition Examination 
Survey 

Parenting Data 

Vital Statistics X X X X X X 

National Household 0 0 X (X) X (X) 

Education Survey 

Supplementary Home 0 0 (X) 0 (X) 0 

Visits 

Preschool Data 

Current Population X 0 X (X) X (X) 

Survey 
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SCHOOL READINESS ASSESSMENT (cont) 



1991 1995 2000 

National State National State National State 



State Counts of 
Participation in 
Special Education 
Programs 

Head Start Program 
Information 

New Program 
Information 



X 



(X) (X) 



(X) 



(X) 



School Entry Data 

School Registration 
Form 

Medical Screening 



0 
0 



0 



0 



(X) 



(X) 



(X) (X) 



(X) 



(X) 



(X) 



(X) 



In-School Data 

Child Development 
Profile 

Parent Report 

Teacher 
Observation 

Portfolios 



0 

0 
0 

0 



0 

0 
0 

0 



(X) 

(X) 
(X) 

(X) 



(X) 

(X) 
(X) 

(X) 



(X) 

(X) 
(X) 

(X) 



(X) 

(X) 
(X) 

(X) 



X = Data Available 

(X) = Data to Be Developed 

0 = Data Not Available 
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VII. The Process: Getting There from Here 

In moving this proposal forward, there arc, we believe, three essential steps. 

• First, several important surveys now being used on a limited basis must be 
expanded so that annual data can be available, nationally as well as State-by-Statc. 

• Second, the most difficult task, designing a new child development profile, will 
require a special research and development effort involving top scholars, key school 
administrators, gifted teachers, and committed parents. In shaping this four-part 
process, extensive and careful work must be done. Further, such research and 
development efforts must be carried on collaboratively through public and private 
effort. Finally, we urge that the effort begin at once so that new instruments and 
procedures can be field tested within 2 years, and made fully operational by 1995. 

• Third, this should not be a top-down process. In developing new instruments, it is 
important to be sensitive to the needs of the various communities concerned with 
early childhood education, and above all, to be responsive to States and lecal 
districts. Not only should state and local representatives — especially teachers — be 
involved in the design process, they should participate in the field test evaluations. 
Above all, the States and local districts should be free to extend the instruments, 
using them to meet their own special needs. 



VIII. Conclusion 

The commitment by the President and Governors to have all children "start school ready to 
learn" signals a striking opportunity and an urgent challenge for the Nation. This national 
goal dramatically underscores the fact that excellence in education begins before school, and 
even before birth itself. We seek to measure progress toward achieving this objective, not to 
penalize children who fall short, but to confront the urgent unfinished agenda of serving all 
students, not just the most advantaged. 
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Chapter 2 
High School Completion 



An Interim Report From the Resource Group on School Completion 



GOAL 2: By the year 2000, the high school graduation rate will increase to at least 
90 percent 

Objectives: 

• The nation must dramatically reduce its dropout rate, and seventy-five percent of 
those students who do drop out will successfully complete a high school degree or 
its equivalent. 

• The gap in high school graduation rates between American students from minority 
backgrounds and their non-minority counterparts will be eliminated. 
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In early 1991, a Resource Group on School Completion was convened by the Panel to 
recommend indicators and strategies for measuring progress toward achieving this goal. 
Members of the group arc as follows: 



Edmond Gordon 
Janet Baldwin 
Eve Bithcr 
Jose Cardenas 

Norccn Lopez 
Steve Niclson 
Aaron Pallas 
Rafael Valdivieso 
Richard Wallace 



Yale University, New Haven, Connecticut (convener) 

GED Testing Service, Washington, D.C. 

Maine Department of Education, Augusta, Maine 

The Intercultural Development Research Association, San 
Antonio, Texas 

Illinois State Board of Education, Springfield, Illinois 
U.S. West Corp., Seattle, Washington 
Michigan State University, East Lansing, Michigan 
Hispanic Policy Development Project, Washington, D.C. 
Pittsburgh Public Schools, Pittsburgh, Pennsylvania 



The Panel and the Goal 2 Resource Group welcome your reactions to the Interim Report on 
School Completion. 
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National Education Goals 
Goal Two 

High School Completion 



I. Statement of Mission 

America has long recognized that schooling is central to educating children for their roles as 
competent, productive adults. Although there are many important educating institutions in our 
society — including the family, the community, the workplace, and the media — schools hold a 
special place, so much so that every State in the Union has laws requiring children and youth 
to attend school. We send our children to school so that they may acquire the skills, 
knowledge, and values necessary to participate fully in adult work and family life and to 
exercise the rights and responsibilities of citizens in a democratic state. 

Because of the importance that we attach to schooling, it is intolerable that nearly half 
a million students per year exit grades 10 through 12 without completing their high school 
program. Our Nation can no longer afford the drain of its resources represented by students 
leaving school before graduation. Responsibility for this tragedy must be shared by the 
education system and the society of which it is a part. And solutions must be pursued with 
unprecedented determination; nothing less than our future as a vital democracy and a vigorous 
economic force is at stake. 

This need appears to be recognized by the Nation's Governors and the President in their 
commitment to the goal of increasing high school graduation rates to at least 90 percent by 
the year 2000. Resource Group 2's approach to the charge of recommending indicators and 
data systems that will monitor progress toward achieving this goal is shaped by four themes 
emerging from our deliberations: (1) the importance of monitoring educational development at 
specific benchmarks such as high school completion; (2) the interrelatedness of the National 
Education Goals and their relation to school reform; (3) the importance of educational equity; 
and (4) the power of information to enhance educational policy and practice. These four 
themes frame our approach to National Education Goal 2 and suggest strategies for assessing 
progress toward its realization. A brief statement of each theme sets the stage for our 
recommendations. 
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A. The Current Importance of the High School Completion Benchmark 

In spite of our interest in what our children know and learn in school, we have rarely 
measured these items directly. Instead, we have relied on the presence or absence of 
credentials such as a high school diploma as a rough guide in assessing whether people 
possess certain skills, knowledge, and values. We recognize, however, that high school 
completion is only a crude proxy for assessing what a person has learned in school. We 
cannot conclude that people who possess these various credentials also possess a specific set 
of skills, knowledge, and values. 

Because of current technical limitations in our abilities to measure directly student 
achievement and citizenship (i.e., the content of National Education Goal 3), we often rely on 
high school completion as a rough indicator of such accomplishments, despite its weaknesses 
in telling us anything of real value about exactly what students have learned. In this sense, 
National Education Goals 2 and 3 are interrelated, because we implicitly treat measures of 
progress toward Goal 2 (high school completion) as measures of progress toward Goal 3 
(student achievement and citizenship). If direct assessments of the skills and competencies of 
America's children were available, thereby measuring progress toward Goal 3, the importance 
of measuring progress toward Goal 2 via the indicator of high school completion might 
recede. We can envision, in fact, a time when measurement of Goal 2 might be redundant or 
superfluous because of progress in measuring Goal 3. But for now, it remains extremely 
important to monitor high school completion rates. 



B. The Interrelate dness of the National Education Goals and Their Relation to School 
Reform 

The National Education Goals are interrelated in another fundamental, yet complex, way. 
Progress toward some of the goals may have consequences for progress toward other goals. 
Our ability to ensure that all children will start school ready to learn (Goal 1) or that all 
schools will be free of drugs and violence and will offer a disciplined environment conducive 
to learning (Goal 6) may influence our success at increasing the high school graduation rate 
(Goal 2). Conversely, our success in increasing the high school graduation rate has important 
implications for adult literacy and lifelong learning (Goal 5). 

These linkages among the goals have an important implication for assessing progress 
toward their achievement. Such connections suggest the value of exploring the potential for 
an integrated indicator system that monitors progress toward all six National Education Goals 
simultaneously. Our conception of an integrated system extends beyond the collating of 
different bits of information in an annual report to a reporting system that takes account of 
the interrelatedness of these goals. For example, we see a need to understand how strategies 
designed to promote progress on Goal 3 (student achievement and citizenship) have 
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consequences, intended or not, for Goal 2 (high school completion). The converse is true as 
well. 

National Education Goals are also contextually related to the national concern for 
school reform. Achieving national goals is not likely to be reached without reforming our 
schools. School reform, in turn, is likely to be powerfully informed by monitoring the 
progress toward achieving these goals. Thus, we view the process of monitoring achievement 
of this goal as an initial step, to be followed by the reorganization and restructuring of critical 
aspects of schooling. Restructuring efforts at all levels of the educational system, from 
kindergarten to professional teacher preparation, may well be necessary if the Nation is to 
dramatically increase its graduation rate, prevent children from dropping out of school at an 
early age, and achieve the other National Education Goals. 

C. The Centrality of Educational Equity 

Much of the interest in school completion and noncompletion stems from the overwhelming 
evidence of the different school completion rates of youth of differing racial or ethnic 
backgrounds. This evidence is explicitly acknowledged in Objective 2 of Goal 2, which 
states that the gap in high school graduation rates between American students from minority 
backgrounds and their nonminority counterparts will be eliminated. 

The importance of educational equity is clear. We know that different groups of children 
and youth have differing patterns of school achievement and participation. Some children 
bom into minority and disadvantaged families and communities may be exposed to 
educational resources that do not prepare them for the curricula that they will encounter in 
school. In addition, the school processes themselves may deny minority and disadvantaged 
children access to appropriate educational resources, programs, and teachers. What we sec 
too often are limitations in the readiness of some children to benefit from what schools do 
traditionally and limitations in the capacities of schools to respond to the needs of some 
students. 

The concern for educational equity reflects the need to reduce the influence of students' 
social status and material resources on educational achievement and competence as citizens. 
The goal of public policy in education is to foster achievement among students regardless of 
race, ethnicity, or language background, thereby eliminating the existing gaps. However, 
effective public policies depend on valid and reliable measures of relevant factors associated 
with high school completion for all students. To achieve the goal to increase levels of high 
school completion among minority and nonminority students, and to eliminate gaps in high 
school completion rates for these subgroups, high school completion must be broadly defined 
and accurately reported by race and ethnic group for geographic regions important to the 
public interest (e.g., the State, the region, and the Nation). Mo-eover, the factors believed to 
influence the high school completion rates of different subgroups (such as parental education) 



21 



Resource Group Interim Reports 



must also be measured and their relationships evaluated to identify educational and social 
interventions that lead to improvements. 

D. The Power of Information to Enhance Educational Policy and Practice 

As previously noted, the National Education Goals should direct the course of educational 
reform in this country by providing a lens to evaluate progress toward successful reform and 
restructuring. We believe that attempts to assess progress toward the goals also send strong 
signals about what we think is important. For this reason, we have embedded the assessment 
of progress toward Goal 2 in a comprehensive information system capable of informing 
educational practice at multiple levels. 

We strongly believe in the power of information to help all of us — business leaders, 
legislators, educators, parents, students, and citizens— to make good decisions about American 
education and to move the education system toward the needed reforms. Education is a 
shared responsibility, not limited to the schools, the States, or the Federal Government. At 
the same time, we recognize the unique responsibility of schools, the most direct providers of 
educational services and programs to our children and youth. We are, therefore, attempting to 
fashion a reporting system that will provide useful information about progress toward Goal 2 
not only to the President and the Governors, but also to schools and school districts, to hel 
them in their day-to-day work with students to progress toward the goal. Developing such a 
system may allow educators to use student information more wisely to achieve a better match 
between students' educational needs and the educational programs and practices to which 
students are exposed. Thus, the information in such a system can be as useful to school 
administrators, teachers, and counselors as to Federal and State policymakers. In this way, 
monitoring national and State progress toward Goal 2 can also facilitate progress toward its 
achievement. 



II. The September 1991 Progress Report 

The 1991 Progress Report can provide limited but important baseline information for 
measuring school completion and dropout rates at both the national and State levels. 



A. Data Reported for the Nation as a Whole 

i. Recommended Measures 

To measure progress toward meeting Goal 2 and its associated objectives, we 
recommend ushig the following five national -level indicators in the September 1991 
Progress Report: 

22 



ERIC 



26 



Resource Group Interim Reports 



Indicator #7 — The proportion of 19- to 20-year-olds and of 24- to 25 l -year -olds 
holding a high school completion credential (data source: Current Population Survey). 
The Current Population Survey (CPS), conducted by the Bureau of the Census, is the 
only source of regularly reported valid and reliable national-level data to measure rates 
of school completion. CPS is a household survey that gathers information from a 
single adult member of the household about everyone in the household. Since 1967, 
CPS has been collecting data in a similar fashion on high school completion rates. 
Completion rates are based on how many complete high school by means of an 
equivalency certificate as well as how many obtain a regular diploma. (Since 1988, 
CPS has collected data on the proportion who complete by means of an equivalency 
certificate of those 24 years old and younger.) 

Completion rates can be calculated for age cohorts using CPS data. The use of 
two age cohorts is recommended so that the phenomenon of rising completion rates 
after the traditional age of completing high school can be computed. This phenomenon 
occurs for two reasons: many of those still enrolled in high school at ages 1" 19, or 20 
do complete, and many dropouts also complete. Completion rates tend to rise gradually 
until the late 20's, but the bulk of the increase occurs by age 25. 

The choice of the lower age range reflects current enrollment patterns. Most 
students graduate from high school at age 17 or 18. However, about one-fourth of 18- 
year-olds are still in high school, and that proportion has been rising over time as 
children start school later and take longer to complete. The age groups arc defined in 
terms of 2-year age ranges, rather than single years, to be consistent with other 
recommended measures. (See below.) 

Indicator #2 — The proportion of 19- to 20-year-olds and of 24- to 25-year-old 
whites, bheks, and Hispanics holding high school completion credentials (data source: 
CPS). For the September 1991 Progress Report, the CPS data on completion rates 
should be calculated for the three largest racial/ethnic categories. These data are 
available since 1967 for whites and blacks and since 1972 for Hispanics. The sample 
sizes in the CPS for American Indians/Alaskan Natives and Asians/Pacific Islanders are 
too small to produce reliable estimates. Furthermore, the sample sizes for blacks and 
Hispanics are too small to produce precise estimates for single years of age. To 
address this problem, 2-year age ranges, for example, 19 to 20 and 24 to 25, arc 
recommended. 

The data should be displayed as separate, mutually exclusive estimates for the 
three groups, rather than as differences between the white rates and those for blacks 
and for Hispanics. The emphasis should be ■ raising the completion rates for 
everyone, not just on liminating differences. 

Indicator #3 — Event dropout rate for 10th to 12th graders ages 15 to 24 (data source: 
CPS). An "event" dropout rate is one of three rates for which nationally representative 
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data are available. This rate represents the proportion of students who leave school 
without completing high school during a single year. We feel this rate is preferable to 
the other dropout statistic available from the CPS— the "status" rate, which represents 
the cumulative proportion of an age group who are out of school and have not 
graduated — because the event rate measures how well schools are presently doing at 
keeping students in school. 2 

Indicator #4 — Cohort dropout rates for 8th to 10th grades (data source: National 
Education Longitudinal Survey: 1988) and 10th to 12th grades (data source: High 
School and Beyond). This third type of dropout rate is a cohort rate where a single 
group of students is followed over time to see what happens to them. The most recent 
cohort data available are for the 1980 sophomore cohort, which is rather old data. 
However, data from the National Education Longitudinal Survey: 1988 (NELS:88) will 
be available by September for 8th to 10th grade dropout rates, which can supplement 
the data from the High School and Beyond (HS&B) survey for 10th to 12th grade rates 
in the 1991 Progress Report. 3 

Indicator #5 — Either the proportion of dropouts from the sophomore class of 1980 
completing high school by 1986 (data source: High School and Beyond) or the number 
of General Educational Development Testing Service certificates (GED's) issued to 17- 
to 24-year-olds as a proportion of the number of 17- to 24-year-old dropouts (data 
source: GED's from American Council on Education and dropouts from CPS). These 
statistics specifically address the objective that 75 percent of dropouts will eventually 
become high school completers. To monitor this condition most accurately requires 
longitudinal data because tracking individuals over rime to see what they do is 
necessary. 

Data from HS&B from 1980 to 1986 are available for this purpose, and 
consideration should be given to reporting these data in the initial Progress Report. An 
alternative approach that is worth investigating is to take advantage of data collected by 
the American Council on Education (ACE) about the number of GED's awarded each 
year by the age of the recipient. The number of GED's awarded to those age 17 to 24 
divided by the number of 17- to 24-year-old dropouts (from CPS) could provide an 
estimate of what percent of dropouts in a single year are receiving GED's. 4 

2. Data Considered But Not Recommended 

The only alternative national data sources on school completion rates are those used to 
calculate the graduation rates in the U.S. Department of Education's Annual State 
Performance Chart (frequently called the Wall Chart). The rate is defined as the 
number of high school graduates as a percentage of the number of ninth-graders 4 
years earlier, both obtained from the National Center for Education Statistics' (NCES) 
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Common Core of Data (CCD). There are a variety of limitations to those data, which 
are discussed at length under State-level measures. Two limitations worthy of special 
note here are that the data do not include participants in private schools, and are not 
available by race and ethnicity. 

3. Special Issues: The Publication of Additional Related Statistics 

Race and ethnicity are not a cause of lower graduation rates (or higher dropout rates). 
Rather, members of racial and ethnic minorities are more likely to be facing 
circumstances making graduation from high school less likely. Such conditions include 
poverty, single-parent families, limited English proficiency, and lack of prior success in 
school. In fact, some research has shown that black-white differences in dropout rates 
are narrowed or eliminated if such factors are considered. Therefore, members of the 
Resource Group believe that measuring this objective should be broader than simply 
focusing on racial and ethnic differences. 

Unfortunately, CPS does not contain data related to these other factors. Therefore, 
we recommend that illustrative data from other sources be used to address this issue 
and that such data be included in the September 1991 Progress Report. We would 
specifically like to sec data from NCES* HS&B study and NELS:88 used for this 
purpose. In addition, these longitudinal surveys have large enough sample sizes so that 
they can be used to produce estimates of school completion and dropout rates for 
Asians and American Indians. Thus, for these two cohorts, rates can be compared for 
all five major racial and ethnic groups. 



B. State-by-State Reporting 
1. Recommended Data 

We recommend that the following three State-level indicators be included in the 
September 1991 Progress Report: 

Indicator #2 — State-by-State data on those holding high school completion 
credentials (data source: CCD). Each year, data are collected by the Federal 
Government on the number of students completing a secondary school credential in 
each State's public schools. These data include number of students receiving a regular 
high school diploma, another type of diploma, a certificate of completion or attendance, 
or a high school equivalency credential. The data, however, are not entirely 
comparable because States have policies that affect who is included in each category. 
In addition, not all States report complete data. These data, however, are usable with 
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certain caveats, and efforts are under way to work with State education agencies to 
further standardize the reporting of these data. 

As previously noted, a school completion rate for each State has been computed in 
recent years by the U.S. Department of Education, using the count of regular diploma 
recipients divided by an adjusted number of students who were in the 9th grade 4 years 
earlier. 3 While this method of computing graduation rates is not universally supported, 
the resulting rates give the only reasonably standardized current estimate of graduation 
rates for each State. 

We specifically recommend that State completion rates be separately computed 
and published using the number of students receiving regular diplomas, other types of 
diplomas, certificates of completion or attendance, and GED's. Together these rates 
would indicate the proportion of students who remain in school and receive some type 
of credential or certificate at the end of the expected period of time. 

Indicator #2 — State-reported graduation data (new State survey). State-by-State 
public school graduation data are not currently reported for different racial and ethnic 
groups through any national surveys. It is possible, however, that some States collect 
graduation statistics for these subgroups. We recommend that States be invited to 
provide completion data for these categories to the extent they are available and that 
such data be published in the September 1991 Progress Report. 

Indicator #3 --State -reported dropout rates (new State survey). State-by-State data 
are not currently available on a comparable dropout statistic although efforts are under 
way to collect these data in the next 2 years. For the September 1991 Progress Report, 
we recommend that States be invited to provide dropout statistics with an explanation 
of the procedures used to collect and report the data. Some States are known to be 
collecting statewide data according to the dropout definition and procedures being pilot 
tested. For these States, baseline data can be obtained for future comparisons. 

2. Special Issues in State Reporting 

It is important to emphasize that the national data in the September 1991 Progress 
Report will reflect all school completers and dropouts, not just public school students. 
State-by-State data, on the other hand, can only report on public school students, at 
least at this time. As a result, care should be taken in comparing the national statistics 
to the individual State statistics. Nationally, approximately 11 percent of the total 
student enrollment in 1980 attended private schools. States varied in the percentage of 
students attending private schools from 1.6 percent to 19.6 percent. 

Even comparing one State's graduation rate with another is problematic because of 
different State policies in granting a high school credential. For example, some States 
give all students an identical diploma, including students in special education programs 
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who do not complete the same requirements completed by students without disabilities. 
Others give different types of diplomas, depending on the type of program completed 
by the students. State policies such as these will have an effect on the number of 
completers being reported. We, therefore, view the State school completer data to be 
published in the September 1991 Progress Report as baseline information to chart State 
progress through the decade. 

HI. Future Data System Improvements 

Our recommendations for improving the national data system to report on progress toward 
Goal 2 include both interim and long-range objectives. We believe that planning for 
long-range improvements should begin as soon as possible, because the recommended system 
involves both technical and sociopolitical considerations that may take several years to 
resolve. At the same time, we recognize that there are several things that can be done in the 
interim to improve the quality of current data collections. These interim activities should be 
pursued simultaneously with the development of the long-range data system. 

A. Interim System Improvement Objectives 

The interim objectives for the national data system to report on progress toward Goal 2 are 
short term and incremental in nature, and they may be rightly characterized as "tinkering 
around the edges. ' Over the next few years, we envision small but meaningful enhancements 
to the existing national and State data reporting systems described earlier. The recommended 
changes can improve the quality of the data gathered without altering the fundamental nature 
of the data collection and reporting mechanisms currently in place. These interim adjustments 
arc a stopgap measure until the more elaborate long-range data system that we are proposing 
becomes a reality. 

In the short run, the Census Bureau's Current Population Survey and NCES 1 Common 
Core of Data will continue to be the data systems of choice for providing national and State 
counts of high school completers and dropouts. 

J. Recommended Interim CPS Improvements 

The Resource Group recommends the following four enhancements to the CPS: 

Ovcrsampling minorities. Even though the CPS surveys 58,000 households on a 
monthly basis, the sample includes too few racial and ethnic minority youth to estimate 
black and Hispanic school completion and dropout rates very accurately. This lack of 
precision makes it difficult to tell if these rates arc actually increasing, declining, or 
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staying constant over time. 6 For this reason, we recommend that the CPS oversample 
racial and ethnic minority households. 

Improving the CPS education items. It is desirable that the CPS items on school 
attendance and completion conform as closely as possible to the categories used in 
administrative record systems like the CCD. To date, there still is much ambiguity in 
the CPS items, which do not distinguish public school completion from private school 
completion and which are unable to distinguish among several classes of high school 
completers (for example, those receiving an alternative high school completion 
credential). While we recognize the importance of maintaining continuity in the 
established data scries, we believe that the need to align our definitions in the national 
data systems may outweigh the need for continuity. 

Assessing the validity of CPS reports. We need better information on the validity of 
CPS reports of educational attainment for two reasons. First, the CPS relies on 
household informants to report educational attainment, and we do not know how 
accurate these proxy reports are. Second, because a stigma is attached to dropping out 
of school before completion, informants may tend to exaggerate school completion. 
We recommend that the Bureau of the Census conduct validity studies to assess the 
agreement of CPS reports with school records of high school completion. 

Assessing the undercount. While the CPS is designed to produce national estimates for 
the civilian noninstitutionalized population, the accuracy of the estimates depends at 
least partly on the quality of the enumeration of households. If certain types of 
households arc underrepresented in the sampling frame, they will also be 
undcrrcpresentcd in the resulting CPS sample itself. Household surveys like the CPS 
tend to undercount poor and minority households; therefore, both the number of high 
school noncomplctcrs and the rate of noncompletion for poor and minority youth may 
be underestimated. We recommend that the Bureau of the Census explore rategics for 
adjusting CPS estimates of high school completion for the probable undercount of poor 
and minority youth. 



2. Recommended Improvements to the Common Core of Data 

NCES, in cooperation with the Council of Chief State School Officers (CCSSO), has 
made great strides in understanding the variability among States in definitions of 
several key concepts, including who is counted as a high school completer, who is 
counted as a high school dropout, and who is counted as a student. We strongly 
recommend that NCES and CCSSO continue their attempts to standardize definitions 
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across States and that they provide technical aid to help States modify their reporting 
categories. 

As previously noted, NCES is currently conducting a feasibility study to determine 
whether obtaining comparable State-by-State dropout rates through the CCD collection 
is possible. If the evaluation of this pilot study indicates that States are able to report 
valid counts of dropouts, we recommend that NCES incorporate dropout counts in the 
annual CCD collection. NCES should provide technical assistance to those States that 
request help in gathering such data. 



B. Long-Term System Improvement Objectives 

Our overall vision for the long-term national data system is rooted in four principles. First, 
the system must be truly national in scope. A system that is simply the aggregate of 
independent State or local school district reports would be unable to track students across 
State or district boundaries. Second, the system must represent the diversity of the American 
education system and of the existing ways of completing high school. That is, the data 
system must respect the autonomy of, and be sensitive to, the different ways States and 
localities define high school completion credentials and the standards for attaining them. 
Third, the reporting system, like all such systems, must produce information that is 
technically sound (i.e., timely, reliable, and valid). A system that produces inaccurate or 
dated information will be of little use to anyone. Fourth, the data system must produce 
information that can be linked to educational policy and practice for students of all ages. 
System data should be useful not only at the national and State level but also to local district 
administrators and school building personnel. Unless potential users of the data system can 
connect the information produced with possible action steps, the data produced are unlikely to 
be used. 

J. Creating a National Student Data Repotting System 

In the long term, we believe that school completion and dropout data should be 
gathered and reported through a national student-level data reporting system. Such a 
system would provide more accurate and comprehensive school completion and dropout 
data than are currently possible. But even more important, we believe that a system of 
this type would provide education service providers and policymakers at all levels with 
the vital information needed to both monitor the health of the educational enterprise 
and tailor student services to individual needs. 

Our vision for a national student data base begins at the building level. The 
systems should place information in the hands of those individuals who can have the 
most immediate impact dfi potential dropouts — building-level staff. In addition, 
school-based information systems can provide information on school completion and 
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school-leaving to the relevant school district and State data reporting systems. In turn, 
the statewide data reporting systems can provide information to the national data 
reporting system. 

It is easy to see the desirable features of such a system. There is ample evidence 
that school staff — administrators, teachers, counselors, and others — frequently lack the 
information on students that they need to make good decisions about matching students 
to courses, educational programs, and social or health services. In its most generic 
form, we see the high school dropout problem as the product of the failure to match 
students appropriately to the educational and social services they need. The 
information contained in student information systems may help teachers, administrators, 
counselors, parents, and students themselves to improve the match between students' 
needs and the services they receive. 

In addition to a relatively brief set of "core" data desired for State and national 
reporting purposes (such as grade level and enrollment status), building-level student 
information systems might contain a wide range of information, including information 
on students' current and previous academic performance and behavior, family 
circumstances, and eligibility for or participation in special programs. These examples, 
however, are intended to be illustrative only. Because student populations and school 
programs differ from school to school, building-level school staff may wish to tailor 
the types of information stored in their student information systems to their distinctive 
needs. It would, therefore, be inappropriate for us to prescribe the relevant information. 

Another key advantage of building-level student information systems is that these 
systems provide an early warning system for detecting potential dropouts. Schools that 
are able to identify a set of factors reliably associated with the risk of dropping out of 
school may be able to allocate or reallocate resources to decrease the potential risk of 
leaving school before completion. 

It is important that our recommendation be clear about exactly what kind of 
information building-level data might be forwarded to the State and national levels. 
We are not proposing a national data system with detailed information about the 
performance and characteristics of individual students. Such a system would raise more 
questions than it would answer. Rather, we are proposing that information at the 
building level be abstracted for use in statewide information systems and that 
information at the State level be abstracted for use in a national data reporting system. 
The national and State data reporting system should not contain any specific 
information about individual children and youth. 

We concluded that monitoring student progress over time and across school 
district and State boundaries would be helped by the use of unique, standardized 
student identifiers across the Nation. We recommend that NCES convene a task force 
to consider developing a national system of student identification numbers, perhaps 
based on students' Social Security numbers. 
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2. Steps in Implementing a National Student Data System 

We harbor few illusions about the speed at which school-based information systems 
can be implemented. In many respects, the technical aspects of system design and 
implementation are much more tractable than the social and political aspects. There 
already are projects across the Nation that are attempting to implement school-based 
information systems at the local district and building level. Many of these projects are 
housed in the Nation's largest public school systems. We hope that the Nation can 
draw on the expertise and experiences of these projects in the planning and 
implementation of a nationwide data reporting system. 

We recommend the following steps toward developing a fully functional nationwide 
student information system: 

• First, the NCES, in conjunction with other organizations such as the Council of 
Chief State School Officers and the Council of Great City Schools, should work 
with a small number of States and districts to develop a set of model student 
information systems. This activity would provide a set of alternative models 
appropriate for States and districts with varying needs and resources. 

• Second, NCES should examine strategies for linking school-based student 
information systems to statewide systems and for linking information from 
statewide systems to a national reporting system. Such strategies might include 
developing "crosswalks" that enable the transfer of data from one system to 
another, planning and developing a network of electronic data transfers, developing 
procedures for maintaining data security and auditing the data, and developing a 
governance structure or structures to support a national data reporting system. 
These activities might be coordinated with the proposed NCES Interstate Student 
Records Transfer Project, which has a similar agenda. 

• Third, NCES should recommend a flexible model or models for statewide and 
school-based student information systems and provide technical assistance to States 
and local school districts in the design and implementation of such information 
systems. 7 

We estimate that the process of developing a fully functioning national student data 
system suitable for State-, district-, and school-level adoption will take approximately 
5 years. 
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J. Special Considerations 

a. State Participation. To be wholly successful, the system that we are proposing 
requires the active participation of all of the States. We recognize, however, that in 
periods of declining resources, States must make difficult tradeoffs between investing in 
programs and investing in information systems. Not all States may have the resources 
to implement statewide student information systems soon. Developing such systems, 
however, may be deemed so crucial that an enhanced Federal role will be required. 
Specifically, the Federal Government may wish to consider cost-sharing arrangements 
that partially subsidize State efforts to establish student record systems. Further, 
adopting compatible systems among the States could be gradual. States can "buy into" 
this system as they develop the relevant resources (including potential Federal support), 
in much the same way that they currently "buy into" the NELS:88 data collections. 

b. Seed for Continued CPS Collection. The recommended national student data 
system is based on the reports of public school enrollments, graduates, and dropouts. 
The system will not provide data on the educational attainments of children and youth 
enrolled in the Nation's private schools. Since approximately one child in ten is enrolled 
in a private school, this is a limitation. As noted earlier, the CPS could be modified to 
produce some relevant data on school completion and dropout rates in both public and 
private schools. 

c. Need for Enhanced Research and Analysis Capabilities. Even when fully 
operational, a national student data system is a cumbersome instrument for basic 
research on the "causes" of dropping out and the effectiveness of policies designed to 
enable a greater number of students to be successful in school. Controlled longitudinal 
sample surveys such as the NELS:88 are the best vehicles for investigating questions 
such as these. Unfortunately, the 8-year gap between these studies is uncomfortably 
long. Especially if there is good reason to believe that rates of school-leaving and 
factors that affect them may be changing over relatively short periods (a reasonable 
assumption in this era of heavy State policy reform), accelerated longitudinal survey 
cycles — perhaps every 2 to 4 years — should be considered. 8 

d. Tracking School -Leavers. The proposed data system relics on school record 
systems for data, rather than on the reports of individuals or other agencies. The 
integrity of this system, therefore, depends on the ability of school-based record 
systems to track students once they have left a particular jurisdiction. This ability is a 
longstanding concern regarding the reliability and validity of administrative data on 
school-leavers, but we believe that considerable progress has been made in accounting 
for such school-leavers. 
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e. Data Burden. We are concerned about the potential for "data burden"— the extent 
to which the activities needed to produce the data required by our proposed reporting 
system become a burden on school staff who must provide such information. There is 
a danger that those who are charged with providing or processing such information will 
simply have this responsibility added to their existing responsibilities, without concern 
for the additional hardships placed on them. School staff must make judgments 
balancing the importance of information with the costs that providing information 
places on the providers. Reallocating resources to meet the data needs for the 
information system may be necessary. 

The burden on school and administrative staff to aggregate, analyze, and report on 
indicators such as school completion and dropout rates may be eased if they were to 
report on 2- to 3-year rather than annual cycles. The frequency of reports from 
student record systems (or from other data sources for that matter) should be related to 
how quickly the phenomenon being measured is thought to change. Historically, 
annual changes in dropout and completion rates have been quite small, with trends only 
becoming apparent over several years. Therefore, as s rule, less frequent data 
aggregations and analyses may ease data burdens while still meeting the needs of 
administrators and policymakers. 

/ An Early Warning Data System. We believe that collecting direct evidence on 
school completion and noncompletion is not the only appropriate strategy for gauging 
progress toward the realization of Goal 2. We particularly recommend developing 
"early warning" indicators that predict the likelihood of not completing high school. If, 
for example, being substantially overage for one's grade or being retained in grade were 
widely recognized as correlates of school noncompletion, then measures of this 
phenomenon at one time could indicate likely outcomes at some later time, barring any 
intervening changes. Trends over time in these early warning indicators might then 
provide evidence on subsequent progress toward achieving Goal 2. While recognizing 
the complexity of this chore, we believe that there is a sufficient body of national 
evidence on the causes and consequences of school completion and dropouts to begin 
such a discussion. 



IV. Summary 

In specifying the National Education Goals, the President and the Governors have 
acknowledged the need to organize our efforts to reform the education system and monitor its 
well-being. The goals and our attempts to measure progress toward them can provide such a 
focus, framing the actions of State and Federal policymakers as they seek to improve 
education. 
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But public education in the United States is shaped at many levels, from the 
schoolhouse, to the statehouse, to the White House. We should not overlook the opportunity 
to allow the National Education Goals and the indicators associated with them to affect the 
actions of individuals at all levels of the education system. Therefore, we have proposed the 
development of a national student data system that will provide feedback to those charged 
with improving education at the local as well as State and national levels. 

The combination of existing data systems and our proposed national student data 
system will provide a powerful tool for assessing the Nation's progress toward Goal 2 — 
increasing the high school completion rate. In the short run, the current data systems, with 
the modifications we recommend, will document the current status of, and future trends in, 
school completion rates at the national and State levels. In the long run, the proposed 
national student data system holds the promise of generating even more useful information 
that might speed progress toward Goal 2. 
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Endnotes 

1. The specific age ranges and grade levels recommended for reporting on this and other 
indicators represent preliminary judgments of the Resource Group. We recommend that further 
technical planning work be undertaken before these determinations are finalized. 

2. The event rate is also preferable for purposes of comparisons with State and local 
administrative record data. Because schools typically calculate event rates, a published national 
"event" dropout statistic can be most readily compared with similar data collected locally. 

3. Data from NELS for 12th graders will become available in 1993 and should be reported in 
that year's Progress Report. 

4. Unfortunately, such data do not provide an estimate of what percent of dropouts ever finish 
by way of a GED nor do they reflect dropouts who complete by any means other than a GED — 
either by obtaining a regular diploma (HS&B suggests one-third of dropout-completers receive 
a regular diploma) or some other type of equivalency certificate (very few). 

5. Adjustments are made for ungraded students and for interstate migration patterns. 

6. For example, while the Bureau of the Census estimates the event dropout rate among 
Hispanics to be 7.9 percent in 1989, because of the small sample size, there is approximately one 
chance in three that the true value is less than 6.6 percent or greater than 9.2 percent, a relatively 
broad range of values. 

7. We have tentatively identified NCES as the primary sponsor of most of these activities, 
although other agencies might take the lead as well. While the development of local and 
statewide information systems is wholly consistent with NCES' mission, the recommended design, 
implementation, and technical assistance efforts would represent a substantial expansion of the 
agency's responsibilities, requiring additional resources. 

8. These more frequent studies need not contain the same volume of information as those on the 
8-year cycle. There may be acceptable tradeoffs between the frequency of data collection and 
the scope of the data collected. 
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Chapter 3 
Student Achievement and Citizenship 



An Interim Report From the Resource Group on 
Student Achievement and Citizenship 



GOAL 3: By the year 2000, American students will leave grades four, eight, and 
twelve having demonstrated competency in challenging subject matter 
including English, mathematics, science, history, and geography; and every 
school in America will ensure that all students learn to use their minds 
well, so they may be prepared for responsible citizenship, further learning, 
and productive employment in our modern economy. 

Objectives: 

• The academic performance of elementary and secondary students will increase 
significantly in every quartile, and the distribution of minority students in each 
level will mo-t closely reflect the student population as a whole. 

• The percentage of students who demonstrate the ability to reason, solve problems, 
apply knowledge, and write and communicate effectively will increase substantially. 

• All students will be involved in activities that promote and demonstrate good 
citizenship, community service, and personal responsibility. 

• The percentage of students who are competent in more than one language will 
substantially increase. 

• All students will be knowledgeable about the diverse cultural heritage of this nation 
and about the world community. 
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In early 1991, a Resource Group on Student Achievement was convened by the Panel to 
recommend indicators and strategies for measuring progress toward achieving this goal. 
Members of the group arc as foliows: 



Lauren Rcsnick 



Gordon Ambach 



Chester E. Finn, Jr. 



Asa Hilliard 



David Hombcck 



Richard P. Mills 



Thomas W. Payzant 
Ch^c Pclton 



Terry K. Peterson 



Marshall S. Smith 



University of Pittsburgh, Pittsburgh, Pennsylvania (convener) 

Council of Chief State School Officers, Washington, D.C. 

Vanderbilt University Educational Excellence Network, 
Washington, D C. 

Georgia State University, Atlanta, Georgia 

Independent Education Consultant, /' *i'< nore, Maryland 

Vermont State Department of Education, Montpelicr, Vermont 

San Diego City Schools, San Diego, California 

San Jose Unified School District, San Jose, California 

South Carolina Business Education Committee, Columbia, South 
Carolina 

Stanford University, Stanford, California 



The Panel and Goal 3 Resource Group welcome your reactions to the Interim Report on 
Student Achievement. 
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National Education Goals 
Goal Three 

Student Achievement and Citizenship 

1. Introduction 

This is a report on the work of the Resource Group for Goal 3 of the National Goals for 
Education. The Resource Group has met twice and conducted several telephone conferences 
during January and February 1991. In developing these recommendations, we consulted with 
a number of people with specialized expertise relevant to our charge. What follows 
represents the consensus of the Resource Group members. 

II. The 1991 Progress Report 

The annual Progress Report should be viewed as a tool for generating discussion of and 
attention to educational achievement goals at every level of the American public, in States 
and localities as well as in national forums. To this end, we suggest a report card format in 
which information is presented in simple and understandable terms. Technical material should 
be included in appendices and briefing documents, not in the main Progress Report. The 
report card format should highlight trendlincs and changes over time. 

Options for the September 1991 Progress Report are limited. There are little data 
available that indicate directly what students know and can do. Nevertheless, a sensible and 
informative report card is possible using existing data along with information from an opinion 
poll that could be conducted between now and Sc| x. 

A. National Reporting 

2. Recommended Measures 

We recommend inclusion of the following data at the national level in the September 
1991 Progress Report: 

a. Achievement Scores from the National Assessment of Educational Progress 
[data source, National Center for Education Statistics (NCES)J. The National 
Assessment of Educational Progress (NAEP) provides the Nation's best currently 
available data on student achievement. It has tracked school achievement since the 
1970's in math, reading, science, and writing and has conducted occasional 
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assessments in other subject matters, including lite* >i re, citizenship, history, the arts, 
geography, and computer competence. For the subjects that have been tested 
repeatedly, a report can be constructed that makes changes and lack of them (mostly 
the latter) visible and easy to understand. The Progress Report can be broken down 
by quartiles (it will show small but steady rises in some subjects for the bottom 
quarter of students, no changes from generally low levels of performance for all 
others); by race and ethnicity (it will show gradual rises among minorities for several 
subjects); by gender; and by public vs. private schools. 

By September it may be possible to report the 1990 mathematics assessment in 
terms of percentages of students meeting the new levels of proficiency (basic, 
proficient, and advanced) that the National Assessment Governing Board (NAGB) is 
trying to define. Serious technical problems- have been encountered in setting these 
levels. If these problems can be resolved in time, we recommend reporting 
percentages of students at each level along with, not as a substitute for, the trendiine 
data just described. It will not be possible to report previous years 1 data in terms of the 
new standards, so no direct comparisons of these standards with those of past years 
will be possible. If proficiency levels cannot be reported this year, we recommend 
continued technical work so that they can be used in subsequent years. The use of the 
new proficiency levels for reporting can provide a major point of focus for national 
discussion and for target-setting for improvements in achievement. 

b. Number of Advanced Placement (AP) Tests Given and Test Scores Earned 
(data source, The College Board). AP tests are examinations given to students who 
take special preparatory courses. They are instructionally relevant examinations of 
high quality. Many colleges offer advanced placement credit to students on the basis 
of AP test scores. Although only a small percentage of students currently take AP 
courses and examinations, the percentage has been growing. An index based on how 
many tests are given, how many are passed, and how many students take at least one 
test can provide a good indication of the extent to which high schools are offering and 
students are opting for challenging academic instruction. Breakdowns by ethnicity and 
race, gender, and public and private schools arc possible. 

c. High School Course Enrollments (likely data source, NCES). Enrollments in 
certain high school courses measure the extent to which students have the opportunity 
to learn important subject matter. Enrollments are also a good indirect measure of the 
quality of academic preparation and motivation in middle schools. National data, based 
on detailed analyses of samples of high school transcripts, are available for 1982 and 
1987. There were some changes in enrollment patterns during that period, apparently 
in response to education reform efforts of the first half of the decade. If the 1991 
Progress Report includes information on these two dates, a baseline for comparison 
with selected years in the 1990's will be established. 
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Wc recommend reporting rates of enrollments in advanced mathematics and 
science courses. Lower-level courses are generally required of everyone, so 
enrollment rates are not useful indicators. 

We also recommend reporting on rates of enrollment in second- and, 
especially, third-year foreign language courses. 

We recommend reporting on the percentage of students who, in 1982 and 1987, 
took the "new basics 1 * series of courses recommended by the 1983 National 
Commission on Educational Excellence, if such an index can be constructed. 

Finally, we recommend, if possible, reporting' on the number of students who 
have studied algebra by the end of 8th grade. In addition to indicating how much math 
a student has learned through the 7th grade, this report is an important "gatekeeper" 
indicator. Students who do not take algebra by 8th grade have little chance of 
completing a high school course that includes calculus, yet many middle schools do 
not currently oiler algebra in 8th grade. 

d. International Student Achievement Comparisons (likely data source, 
International Evaluation of Educational Achievement (IEA)J. We believe that 
appropriate international comparison data in as many subjects as possible are essential. 
The Resource Group for Goal 4 is investigating sources of comparative data for 
mathematics and science. International comparison studies by IEA are being 
developed in mathematics, science, reading, computer literacy, and early childhood 
education. Several of our recommendations call for deliberate efforts to benchmark 
educational standards against those of other nations. 

e. National Poll on Satisfaction of the Education System's Clients. These are the 
only kind of new data that could be made available for September 1991. We 
recommend considering a poll of clients of the education system (employers, 
postsccondary educational institutions, parents, and students several years after leaving 
school). Such a poll, repeated annually or biannually, would help to track changes in 
satisfaction with educational achievement over the decade. 

2. Rejected Measures 

We recommend against use of the following measures at the national level in the 
September 1991 Progress Report: 

a. SAT and ACT Tests. Because so little good data are available for tracking 
achievement in American schools, these college entrance tests are often used for 
comparing States or schools. We recommend against continuing this practice. Each of 
these tests is taken only by college-bound students in certain States. This limitation 
makes for a variable and nonrepresentative sample. More important, the tests do not 
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directly measure what is taught in schools. They are, therefore, not good measures of 
school achievement. 

b. College Board Achievement Tests. These tests more directly represent high 
school course content than either the SAT or ACT. However, they are not widely 
used at this time, so they will not provide a very reliable sample of school 
achievement. 

c. International Baccalaureate. This high-standard r urination works to an 
international standard. It is tied to course content, and students actively study for the 
exam. However, since only 2,000 to 3,000 American students take this exam each 
year, it will not provide a useful indicator for the Progress Report. 

d. Military Screening Exams. Military screening tests (AFQT and ASVAB) are 
taken by many American youth, sometimes in their high schools. We recommend 
against using these tests in the Progress Report for the same reasons that we do not 
like the use of SAT and ACT for this purpose. A variable and limited population takes 
the AFQT and ASVAB, and they are not tied directly to any instructional program. 

3. Data Alternatives and Unresolved Issues 

a. Citizenship, Community Service, and Personal Responsibility. We have not yet 
been able to find reliable indicators of the extent to which this objective is being met. 

b. Knowledge of Diverse Cultural Heritage and World Community. We have not 
yet been able to find reliable indicators of this objective. 

B. State-by-State Reporting 

Data available for State-by-Statc comparisons arc even more limited than for the Nation as a 
whole. We considered both data that would be comparable across States and a possible 
collection of State self-reports that are not completely standardized. The State-by-State 1991 
Progress Report that we are recommending will not be rich. However, we think that it would 
be better to invest in building an excellent end-of-decade system than to attempt to create 
• indices based on the current data. 



1. Recommended Measures 

We recommend including the following data at the State level in the September 1991 
Progress Report: 
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a. NAEP State -by -State Data, In 1990, 37 States participated in a trial use of 
NAEP for Statc-by-Statc comparison in eighth-grade mathematics. The quality of the 
experiment is currently being evaluated by a National Academy of Sciences panel. 
Assuming a favorable evaluation and successful resolution of the problems related to 
the proposed NAGB levels of proficiency, we recommend reporting these data. We 
note that the interest of this data will be primarily to establish initial reference points 
for future trendline data and to establish expectations for future Statc-by-Statc NAEP 
reporting. 

b. Advanced Placement Tests. State-by-State data are available on who takes AP 
tests and their scores. These data should be included in the 1991 Progress Report. 

c. Course Enrollments. Course enrollment data paralleling the national data 
should be reported for the States. 

d. Individual State Reports. Because comparable Statc-by-Statc data for 1991 
will necessarily be sparse, we suggest the possibility of inviting each State to 
contribute a brief report on the State's system of monitoring school achievement 
results, related to some self-chosen reform effort. The intent of this latter invitation is 
to give each State a chance to report on some positive activities in the State and to 
highlight systemic reform efforts. If individual State reports are invited, considerable 
thought will need to be given to designing the common questions and the forms of 
presentation. (See Section IV.B. for some possible questions.) 

2. Rejected Measures 

We considered, but recommend against, the following additional possibilities: 

a. Norm -referenced Achievement Tests. Tnese are used by many States as part of 
their state assessment programs. However, different tests are used by different States. 
The "grade-lever reporting used does not necessarily imply a common achievement 
standard because the tests have not been equated to one another. Furthermore, the 
tests have been designed to compare students, not to measure how well they do against 
a planned achievement standard. Thus, they invite "Lake Wobegon" reporting (all 
States above average) and provide no means of informing the public as to what 
substantive competencies are represented by different scores. 

b. SAT, ACT, College Board Achievement Tests, AFQT, International 
Baccalaureate. We recommend against the use of these tests at the State level 

for the same reasons that we recommended against including them in the national 
report. 
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III. Future Progress Reports: Long-Term Data System Design 

The panel's major task is to lay out a strategy for long-term assessment of achievement 
outcomes. As one can see from the foregoing recommendations for the 1991 Progress Report, 
there is a paucity of decent outcome data presently available. We recommend the 
development of a nationwide assessment system based on world-class standards that can 
ultimately be used to transform American education. 

A. The End-of-Decade Nationwide Assessment System 

1. Rationale and General Approach 

Observations of examining practice in America's competitor nations show that several 
different structures are possible for a national examination system. Some countries use 
a single examination for all students, although exam grading is usually conducted 
locally or regionally. Other countries have developed systems in which several 
different authorities administer their own exams. The countries have a variety of 
methods for ensuring that exams and grades are based on an equivalent standard. 
Apparently, some degree of decentralization has been helpful in keeping educators 
informed of and invested in the examination process and in promoting innovation in 
examination procedures. Such investment is the key to the effectiveness of an 
examining system in creating and maintaining high educational standards. 

Recognizing the desirability of some degree of decentralization in an 
examination system for a country as large and diverse as ours, the panel has been 
discussing a "cluster" model for a nationwide assessment system. In this model, States 
or clusters of States would develop shared curriculum frameworks and exams. The 
cluster exams would be calibrated to a national standard. A full assessment system 
would include both program assessments, designed to monitor the overall effectiveness 
of the education system, and individual student assessments, designed to motivate 
student and teacher efforts to a high level of academic achievement. Both assessments 
should reflect the same national goals for educational achievement and should, 
therefore, be based on a carefully developed national educational standards framework. 



2. Building the System 

In order to create this proposed national assessment system, three major tasks must be 
successfully accomplished. First, a national educational standards framework must be 
developed. Second, when the framework is completed, it must serve as the basis for 
designing a set of national anchor examinations. Third, criteria must be created for 
evaluating student performance on the anchor exams. 
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a. Creating a National Educational Standards Framework. One of the panel's 
most vital tasks is to initiate discussion of and make recommendations to the 
Governors, the President, and the Congress on an appropriate process for creating a 
national educational standards framework. The standard-setting process itself, if 
carried out in a broadly consultative manner, can be part of the educational renewal 
sought by the panel, producing greater interest in and commitment to educational 
achievement among all citizens. To establish a national education standards 
framework will require appropriate authorization of a nationwide entity to provide 
leadership for the task. For purposes of illustrating the type of entity and the role for 
such a body, the following description of a hypothetical "Board" is provided. Such a 
board would establish procedures for a process of standards setting designed to engage 
Americans of all social groups ami in all parts of the Nation in a substantive 
consideration of what they want young people to know and be able to do as a result of 
their schooling years. The board itself should broadly represent the various 
constituencies with an interest in education— parents, business and labor, 
policymakers, educators, community advocates, and curriculum associations. 

There are several different ways in which a consultative standard-setting 
process could be conducted. For example, the process could begin in the States, with 
each State conducting discussions among its citizens of educational goals and 
standards and deciding on its own set of achievement objectives. The national board 
could then be responsible for identifying the common elements in the different 
clusters' objectives. These common elements could become the basis for a national 
standards framework. Alternatively, the process could begin with the board proposing 
knowledge and skill objectives for consideration by the States. Neither of these options 
in its pure form seems likely to produce a high-quality national standard toward which 
schools throughout the country could work. The first, the begin-in-the-Statcs 
approach, might produce such differences in language and substance that it would be 
difficult to find enough commonality for the national standard. Furthermore, it would 
deny to individual States or clusters, the guidance of the substantial work on defining 
educational goals and objectives that has been done by national curriculum 
associations and other groups. Hie second, the begin-at-the-center approach, is likely 
to produce resistance rather than willing adoption by States and local school districts. 
We think a mixed strategy in which discussion moves back and forth between the 
central board and the States several times is most likely to be effective. The process 
might work something like this: 

1. First, the board would collect and make available to States the curriculum 
standards or guidelines that have been produced by various national curriculum 
study groups in the nation [e.g., the National Council for Teachers of 
Mathematics (NCTM) for mathematics, the American Association for the 
Advancement of Science (AAAS) for science and technology, and the Bradley 
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Commission for history], along with current State curriculum frameworks and, 
perhaps, curriculum frameworks from several other countries. States would set 
up procedures for review and comment on these frameworks by groups of 
educators and citizens, preferably at the level of local schools or districts. The 
proposed frameworks would be used to stimulate and guide discussion. All 
segments of the population interested in education — parents, teachers and other 
educators, employers, and community advocacy groups — would be invited to 
participate in this process. 

2. Based on these discussions, each State would compile a report on the 
interests and preferences of its citizens with respect to educational content and 
skills. These reports would be returned to the national board and might also be 
widely circulated within the State. We recognize that many States have already 
conducted consultations of this kind. Such States might wish to submit their 
existing frameworks rather than conducting a new consultative process, or they 
might wish to use the opportunity to compare their own frameworks with those 
offered by the board. 

3. Next, on the basis of the State reports, the board would prepare a draft 
proposal for national knowledge and skill objectives. Representatives from all 
education constituencies would participate at the national level, as they had in 
the individual States or clusters. This draft would be returned to the States, 
which would arrange for review by citizen groups, as in the preceding step. 

4. Finally, the board would revise the draft, thus creating a detailed set of 
knowledge and skill objectives for the Nation. This draft would be offered to 
States and clusters for adoption. 

b. Putting the Standards to Work: National Anchor Examinations. Although the 
cluster concept of a nationwide assessment system leaves authority for selecting and 
administering examinations to the States, some method is needed to ensure that the 
different examinations are in fact meeting the same standard. The most likely way of 
doing this in the United States is to create a set of national anchor exams in various 
discipline and skill areas. These examinations would directly reflect the national 
standards. They would provide a calibration standard against which the cluster exams 
could be evaluated. Technical work on calibration procedures is under way in several 
projects. This recommendation assumes that appropriate calibration procedures can be 
developed. It might also be possible to permit States or clusters to administer the 
anchor exams directly to their students. 

The fundamental reason for introducing a system in which individual students 
arc examined is the belief that such examinations can provide focused targets for study 
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and instruction, thus raising achievement levels in the Nation. To serve this purpose, 
•examinations must function as an integral part of the educational system, capable of 
setting a clear standard toward which students and teachers can work. This goal 
dictates the kinds of assessments that will be needed. They must be assessments 
focused on high levels of achievement (thinking and reasoning, not routinized skills), 
assessments tied to curriculum goals or frameworks, assessments designed to be 
studied for and taught to. No broad assessment system that meets all three of these 
criteria and is designed for our entire school population is available today. Building 
such systems will require substantial effort and resources over the decade. As for 
initial standard setting, we think an iterative process, in which states and the national 
board work together, will produce the best results. Here is a possible process: 

The national standards framework is a necessary first step in defining 
educational standards. However, descriptions alone cannot fully define or communicate 
the kinds of skills and knowledge that are intended. To understand the skill and 
knowledge goals, educators, parents, and other groups of citizens will need examples 
of the kinds of performances that will be taken as evidence that students have met the 
objectives. Developing these examples will also serve as a first step in creating 
performance assessments for the objectives. 

The first stage in developing examples would be to collect and generate 
appropriate performance tasks for the various achievement objectives. This stage could 
be initiated by the States and clusters or by the board. If the States take the lead, more 
individuals would be involved, producing a richer array of candidate tasks. States' 
work on generating assessment tasks could serve simultaneously as part of the proc:ss 
of establishing cluster exams. The national board could stimulate the process by 
providing examples of tasks (including some borrowed from other countries) and 
technical assistance, but the board would not attempt to impose details of a process or 
point of view. 

The next stage consists of selecting those tasks that will define the national 
standard. At this point, the board would need to take the lead, again in a broadly 
representative process. 

c. Setting Grading Criteria. The process of setting standards is not really 
complete until criteria have been set for grading performances on the defining, anchor 
exam tasks. The national board's responsibility is to assure that any performances 
included in the anchor exam be judged with acceptable degrees of reliability. This task 
would require actually administering the anchor examinations and conducting grading 
exercises. The ultimate product would be both a reliable anchor examination to be 
used as a calibration standard and a means of communicating to the Nation what our 
national educational standards are. 
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3. Major Issues and Alternatives in the Long-Term National Assessment System 

a. The Continuation ofNAEP. The Nation needs an independent education 
monitoring system, at ieast until the new individual student examination system is 
being developed NAEP can continue to play this role, but only if NAEP is not turned 
into a "high stakes" test that students are trained to pass. Maintained as an independent 
indicator test, NAEP can provide an important "audit" function on the new assessment 
system as it is developed. 

Therefore, we recommend that NAEP be maintained in its current, matrix 
sampling form, at least until the new assessment system is in place and well accepted. 
At that time, possibilities for merging NAEP and the new system can be considered. 
Consideration should be given — as the National Governors' Association has 
recommended — to authorizing states and local authorities to use the NAEP tests, to 
the extent this can be done reliably within the matrix sampling system. In future years, 
NAEP tests should be constructed on the basis of the new national standards, but the 
tests should also be designed to permit trendlines and comparisons with past years. 

We further recommend investigating ways of calibrating NAEP to international 
examinations or to the examinations of other countries. In this way, NAEP could serve 
as an instrument for monitoring America's progress with respect to other nations. 

b. Examinations in Foreign Languages. High immigration rates require that 
explicit attention be given to the question of how to include children of limited 
English proficiency in a system of nationwide assessment. We assume that all 
American children will be expected to learn English thoroughly. We therefore 
recommend that all children be examined in communication skills — oral and written — 
in English. Information on the number of years that normative-born children have 
been in the United States should be collected and used in interpreting the success of 
schools in meeting communication skill standards. 

At the same time, the native language skills of immigrant children constitute a 
potential resource to the nation; such children will more easily than others be able to 
meet the objective of showing competence in two languages. There is evidence that 
unless schools show overt evidence of valuing native languages, many children will 
refuse to continue using them. We therefore recommend that children's communication 
competencies be assessed in two languages, beginning in elementary school. We 
expect that this practice will encourage earlier and more intensive foreign language 
instruction for native English speakers and will preserve native language capacity 
among immigrant children. 

In subjects other than language, (math, science, etc.) testing in English can 
mask real competencies. To avoid this problem and present a true picture of 
achievement levels, consideration might be given to testing these subjects in the child's 
language of instruction. 
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IV. An Interim Reporting System 

The new assessment system that we have recommended may take most of the decade to put 
into place in its entirety. In the interim, a reporting system can be maintained that includes all 
of the elements recommended for the 1991 Progress Report along with expanded data from 
NAEP. The Progress Report during the interim years should also include reports from the 
States on progress related to systemic educational improvement. 

A. NAEP Expansion 

NAEP would provide a more complete monitoring system if it included regular, periodic 
(perhaps every 2 years) testing in all major subjects (a basic list would include mathematics, 
science, English, history, and geography; a more extended list might add literature, social 
sciences, and the arts) at 4th, 8th, and 12th grades. NAEP results should be reported for 
individual States and perhaps school districts. (See our earlier recommendation.) We note that 
State-level reporting after 1992 will require congressional action. Furthermore, even if 
technical difficulties in reporting proficiency levels cannot be solved by this fall, we strongly 
urge that appropriate work be done to permit such reporting in subsequent years. This form 
of reporting would allow States and the Nation to set meaningful targets for improvement in 
achievement levels. 

B. Collection of Data on Systemic State Reforms 

In its 1990 report, "Educating America: State Strategies for Achieving the National Education 
Goals,- the Task Force on Education of the National Governors' Association (NGA) called for 
radical changes in the structure and functioning of State and local education systems. The 
Task Force outlined a set of systemic reform strategies to effect the changes described as 
necessary. These strategies are designed to help motivate the Nation, the States, communities, 
and schools to restructure the practice of education sufficiently that our students will know 
and be able to do those things that will permit them to be effective citizens and productive 
workers. The development of new assessment tools was put forward as a top priority in 
effecting change in this country's education system. The report recognized that an 
examination system based on world-class standards is a key element in a plan for revitalizing 
American public education, but also that an examination system alone is not enough. Each 
State must also initiate a systemic strategy aimed at ensuring that all students in the State be 
given the maximum opportunity to succeed. 

We recommend that the interim 1991 Progress Report include information collected 
from each State that will indicate the State's progress toward systemic education reform. The 
reform strategies outlined in "Educating America" are echoed in a number of reports and 
policy positions. A consensus appears to be emerging as to what components of a model of 
change are most likely to significantly increase student achievement by all children. The 
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following are examples of the kinds of indicators of a strong systemic change strategy that 
might be included in the 1991 Progress Report: 

• Has the State taken actions that demonstrate the conviction that all children can 
learn? Such actions could include antitracking initiatives, increased proportions of 
disabled students being successfully educated in regular classrooms, and data 
demonstrating that the achievement gaps between ethnic and gender groups are 
narrowing. 

• Has the State adopted student achievement goals and targets that reflect high 
expectations in the disciplines and qualities, such as problem solving, critical 
thinking and integration of knowledge that cuts across the disciplines? If not, has 
the State initiated a participatory process within the State that will lead to the 
establishment of such goals and targets and their acceptance by parents, educators, 
and citizens? 

• Has the State developed curriculum frameworks that embody the outcome 
achievement targets in at least the curriculum areas of national Goals 3 and 4? 

• Has the State identified and/or embarked on developing assessment strategies that 
are as rich as the outcomes that they wish all their children to achieve? Do the 
assessment strategies reflect the achievement goals and targets established in the 
curriculum frameworks? 

- Has the State developed a system of accountability that provides powerful rewards 
to schools and school staffs when students succeed in meeting target outcomes or 
• are moving satisfactorily toward meeting them? Does the accountability system 
provide aggressive assistance of a variety of kinds to unsuccessful schools and 
school staffs and significant corrective action in the face of persistent failure? 

• Has the State established a strategy for teacher professional development that 
ensures that all teachers are well prepared to teach effectively the content necessary 
for students to succeed on the achievement examinations? Have the State and local 
systems developed a strategy to ensure that all continuing teachers are prepared to 
teach the material in the achievement content frameworks? 

• Has the State created a structure within which teachers and other local school 
professionals are given the freedom and responsibility to best figure out how to 
achieve the goals and targets established at the State level? Has the State strategy 
created a context in which decisionmaking power is moved down the bureaucratic 
pipeline in a manner that aligns accountability and control of instructional^ related 
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decisions? 

- Is the State systemic strategy in all of its components designed to place a premium 
on the achievement of minority, poor, limited-English-proficient, disabled students 
and any others with whom schools fail in disproportionate numbers? 

• Does the State provide a quality, developmental^ appropriate prekindergarten 
program for at least its low-income 4 year olds? 

• Has the State developed a coordinated system through which the health and social 
service barriers to student achievement are being reduced? 

• What evidence is there that the state sees the elements of its change strategy as 
integrated or systemic? Are the parts of the system aligned with one another? For 
example, is teacher training directed at the curriculum framework? Is the reward, 
technical assistance, and penalty system related to the outcomes? Will the 
assessment system measure the outcomes? 

If a State has defined a fundamentally different systemic State reform strategy, the 
opportunity should be provided for the State to describe its elements. 

We recommend that information be collected annually on these items. In addition, we 
recommend that a system be established whereby an independent and diverse citizens' group 
in each State examine the evidence behind the answers to these questions every other year to 
ensure that the process is not simply a checklist effort lacking depth and quality. 

V. Tasks, Schedules, and Timetables for Implementing Interim and 
Long-Term National Assessment Systems 

A. Grade Levels and Subject Matter for the Anchor Examinations 

The National Goals for Education call for American students leaving grades 4, 8, and 12 to 
demonstrate high levels of competency in school subject matters. These three points of 
achievement measurement reflect NAEP score availability. We assume that NAEP will 
continue to report at these grade levels. However, a different pattern of examination 
milestones might be more aptly incorporated into the examination portion of a nationwide 
assessment system designed to motivate and organize studying and teaching to a high 
standard. 

Careful thought and considerable consultation will be needed to select two or three 
key points in the grades kindergarten through 12 school cycle for examination. Points of 
examining should be related to points of transition or decisionmaking in a student's career and 
should permit school staffs to organize coherent, multiyear educational programs preparing 
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students for the examinations. Particularly for secondary students, consideration should be 
given to permitting students to accumulate examination credits over several years rather than 
sitting for a single pass-or-fail test. This method could accommodate different rates of 
learning, yet hold all students to a high standard of achievement. We believe that decisions 
on timing of examinations will best be made by the board, building on information gathered 
during the process of establishing the educational standards framework. However, the 
Resource Group could, if requested, provide an analysis of the issues and choices the board 
should consider. 

B. Timelines and Costs for Development of the Nationwide Assessment System. 

Our Resource Group recognizes the urgency of putting the new assessment system in place as 
quickly as possible as part of a national move toward excellence in education. We have 
outlined a three-step plan of action, beginning with development of a national educational 
standards framework that can then serve as a basis for designing a set of national anchor 
examinations and grading standards. 

There are several possibilities for organizing the framework and standard-setting 
process. Each has advantages and disadvantages, with inherent implications for the schedule 
on which the examination system could come into use. We estimate that once agreement on 
the standards framework for a given subject has been reached, it will require 2 or 3 years to 
develop and validate an anchor examination. This requirement means that if the process of 
creating the standards framework were to begin this year, the first examinations could be 
ready for use by mid-decade. Additional examinations, for a more extensive system, could 
be phased in over the remainder of the decade. The Resource Group is ready, at the panel's 
request, to develop alternatives and recommendations for expediting this process of exam 
development. 

There are two parts to the cost projection question. The first is the cost of developing 
the system and initially putting it in place. These estimates need to be developed in 
conjunction with establishing a process and timelines. The second is the operating cost for 
the kind of examination system we recommend once it is up and running. There are 
objections that the kind of assessment our recommendations call for will be impossibly 
expensive. These claims are based on an assumption that all costs of setting, administering, 
and grading exams would be add-on costs to operating school budgets. We think there is 
another possibility, one in which much of the cost would be absorbed in operating budgets 
because participation in examining would be seen as a part of the job definition for 
professional teachers. The Resource Group is ready to work on this question at the panel's 
request. We would want to consult with individuals outside our group to develop estimates of 
real costs under different assumptions. 
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Chapter 4 
Science and Mathematics 



An Interim Report From the Resource Group on Science and Mathematics 



GOAL 4: By the year 2000, VS. students will be first in the world in science and 
mathematics achievement 



Objectives: 

• Math and science education will be strengthened throughout the system, 
especially in the early grades. 

• The number of teachers with a substantive background in mathematics and 
science will increase by 50 percent. 

• The number of U.S. undergraduate and graduate students, especially women and 
minorities, who complete degrees in mathematics, science, and engineering will 
increase significantly. 
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In early 1991, a Resource Group on Science and Mathematics was convened by the Panel 
to recommend indicators and strategies for measuring progress toward achieving this goal. 
Members of the group are is follows: 



Alvin Trivelpiece 



Iris Carl 



Oak Ridge National Laboratory, Oak Ridge, Tennessee 
(convener) 

Houston Independent School District, Houston, Texas 



Linda Darling-Hammond Columbia University, New York, New York 



Edward Hacrtcl 



Ken Lay 



Steve Lcinwand 



Stanford University, Palo Alto, California 
IBM, Armonk, New York 

Connecticut State Department of Education, Hartford, 
Connecticut 



Michael Nettles 
Scnta Raizen 

Ramsay Scldcn 



University of Tennessee, Knoxville, Tennessee 

National Center for Improving Science Education, 
Washington, D.C. 

Council of Chief State School Officers, Washhjton, D.C. 



The Panel and the Goal 4 Resource Group welcome your reactions to the Interim Report 
on Science and Mathematics. 
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National Education Goals 
Goal Four 

Science and Mathematics 



I. Statement of Mission 

Much has been written about the declining state of science and mathematics education in 
the United States. Most obseivcrs agree that extensive reform is needed. It is Htical to 
the future of our country that we create an equitable, effective educational system that will 
allow our children to compete successfully in science and mathematics with students of 
other nations. At stake is more than economic superiority or a competitive edge in 
commerce. The quality of American life is endangered when people cannot make 
informed social and political decisions on issues that arc increasingly shaped by science, 
mathematics, and technology. The Nation needs, in addition to a steady supply of 
scientists, mathematicians, and engineers, a scientifically literate and numerate population. 

We must encourage students to express their natural curiosity about the world and 
not allow an archaic educational system to stifle their imaginations and tell them that these 
subjects are "too difficult" and only for an elite group. Too few of our young citizens are 
choosing science and mathematics careers, and too few are learning what is needed as 
entry-level knowledge for many jobs. One result is that the private sector must invest 
substantial resources to correct the problems handed down to them by our failing 
educational system. 

Achieving the goal of "first in the world" in science and mathematics achievement 
by the year 2000 is an enormous challenge. To meet the challenge, the United States must 
develop an infrastructure that creates and nurtures a world-class education system. The 
strategic components of such a system require us to develop 

• Supportive public attitudes and expectations that place a high value on scientific 
literacy and mathematics numeracy for all members of society. 

• Widespread belief that all children can and must succeed in school science and 
mathematics. 

• Teacher professionalism that is first rate. 
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• Fairness and equity for all children. Equity for all requires excellence for all; 
neither will thrive without high expectations. 

• National curriculum standards for schools, districts, and States to use to build 
their science and mathematicsprograms. 

• Tests, assessments, and accountability systems that measure the valued 
knowledge, skills, and processes promulgated by the national curriculum 
standards. 

• Instructional materials and equipment and other learning tools for students who 
will spend most of their lives in a world that will continually be shaped by 
mathematics, science, and technology. 

We have less than a decade to reach our goal. Baselines must be established for 
data that describe the products of our current system so that we know where we are and 
how to progress. Time is critical, yet we must develop sound procedures and techniques 
so that our assessments can inform our efforts to improve. 



II. Structure of the Report 

When the 22nd Annual Gallup Poll of the Public's Attitudes Toward the Public Schools 
asked about the National Educational Goals, Goal 4 fared very poorly. Among the six 
goals it was given the lowest priority (19 percent assigning low or very low priority) and 
the second lowest likelihood of attainment (65 percent saying unlikely or very unlikely). 
These public perceptions underscore why science and mathematics achievement appears as 
a goal separate from, and in addition to, Goal 3 on student achievement across the 
curriculum. 

Because oui .Jturc depends so directly on a steady flow of strong and imaginative 
research leaders, as well as a quantitatively and scientifically literate workforce and 
populace, we must overcome public doubts about the importance of Goal 4 and its 
attainability. In fact, traditional, incremental, small-scale change will be insufficient to 
make U.S. students first in the world in science and mathematics. Only bold, systemic 
changes coupled with new and different measures of success will allow us to attain this 
ambitious and critical goal. 

Given the complexity of systemwide change, progress toward attaining this 
national goal must be charted on multiple measures. Moreover, the outcome-oriented 
nature of the goal and the clarifying objectives make it critical to monitor and report 
progress in science and mathematics in a multifacetcd manner. 
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We propose that progress toward the attainment of Goal * and its accompanying 
objectives be monitored and reported on the basis of four sets of outcome measures: 

1. student achievement in science and mathematics; 

2. the infrastructure of mathematicsand science education; 

3. the training of teachers in science and mathematics; and 

4. enrollment and other mathematics and science pipeline data. 

This framework provides a clear, useful structure for presenting the September 
1991 Progress Report. However, it also serves to identify major gaps in currently 
available data and helps develop plans to gather more and better information for future 
reports. 



III. Measuring Progress Toward Goal 4 
A. Measuring Academic Achievement 

Progress toward Goal 4 can only be measured directly through intc Clonal 
assessments comparing the academic achievement of U.S. students in mathematics 
and science to that of students in other countries using the same instruments (in 
translation). To sustain such international assessments throughout the decade will 
require continual cooperation from other countries. O.ie important factor in 
assuring such cooperation is to collect data that will be useful to all participating 
countries. This step will require extensive collaboration with other countries in 
the design of the assessment instruments and studies. The international 
assessments should do more than simply provide national comparisons of the 
academic achievement of students in mathematics and science. They should also 
collect sufficient information to provide a context for understanding the sources of 
national differences in academic achievement. 

The achievement tests used in the international assessments should reflect 
the best thinking about the use of alternative assessments of higher order thought 
processes. The international assessments should be equated over time and where 
possible, across studies, so that international and national trendlmcs could be 
developed. Global trends in mathematics and science performance levels provide 
an important context for measuring and interpreting U.S. progress toward Goal 4. 
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1. Data Reported for the Nation as a Whole 

There will be no new data available for the 1991 Progress Report, which can 
elaborate the results from the three international studies of science and 
mathematics achievement conducted during the 1980s, including the second 
international mathematics study (in 1982), the second international science study 
(in 1986), and the international assessment of educational progress (in 1988), 
covering both mathematics and science. 

Public reporting of these data would help establish a baseline of where the 
United States stands compared with other countries. It is important to report data 
for all students. For the older students (upper secondary school), countries vary 
greatly in the percentage of students who remain in school. Thus, selectivity 
would need to be reported along with mean score levels for older students. It is 
probably better to report with countries as the unit and not to report data by 
province or language group. A decision must be made whether to report only 
data from industrialized countries or to include data from all participating 
countries. 

There were 23 countries participating in the second international science 
study, 22 countries in the second international mathematics study, and 6 countries 
in the 1988 international assessment of educational progress. 

2. Data Reported for States 

We recommend the publication of State data for those States that participated in 
the international science and mathematics studies. 

3. Data for Future Progress Reports 

The planning for a second International Assessment of Educational Progress 
(IAEP-91) by the Educational Testing Service (ETS) is well under way. The 
study will include approximately 20 countries that will make assessments in 
science and mathematics. The data will be collected in 1991, and the results 
should be available in 1992 in time for the 1992 Progress Report. 

The planning for the third international mathematics and science study to be 
conducted by the International Association for Evaluation of Educational 
Achievement (1EA) has begun. This study is to be conducted in 1994, and the 
initial reports of the results arc to be available in 1995. 

Data collection for a fourth IEA international mathematics and science study 
is scheduled to begin in 1998, and the initial reports of the results arc to be 
available in 1999. 
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In addition, appropriate assessment instruments need to be developed for 
international comparison of America's college and university mathematicsand 
science curricula and for measuring the mathematicsand science achievements and 
abilities of America's college graduates. 



4. Measurement Strategies 

The following are some measurement strategies available for implementing the 
above recommendations: 

• Administer periodically a common test to representative samples of students in 
different countries. There needs to be an evaluation of the alignment of such a 
cross-national test to national curriculum and instruction. 

• Compare the content and performance on examinations in use in different 
countries, then establish equating procedures to determine the relative emphasis 
of instruction and learning in different countries. 

• Explore the utility of allowing States to supplement international assessments 
with State samples so that States can compare the performance of their students 
to the performance of students in other countries. 

• Investigate the use of different formats of the items in the assessment including, 
for example, the use of objective multiple choice, written essays, open-ended 
problems, and performance assessments. 

B. Measuring the Strength of Mathematics and Science Education 

Objective 1: Mathematics and science education will be strength?*?*! 
throughout the system, especially in the early grades. 

This objective clearly entails a systemwide approach to improving science and mathematics 
education. Critical components that must be addressed in concert include 

• Standards for curriculum and learning goals. 

• Instructional conditions, including school organization, curriculum materials and 
facilities, and instructional practices. 
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- Tests and assessments of st jdent achievement and accountability systems. 

* Provision and allocation of resources. 

* Public attitudes and expectations. 

Improvement in each of these components is necessary to achieve the 
strengthening of mathematics and science education throughout ,he system, and the 
improvement must ensure equal access for all students to high-quality mathematics and 
science education. Consequently, a complete progress report, whether at the national or 
State level, would track progress on each component and allow disaggregation by gender, 
ethnicity, socioeconomic status, primary language, and disabling conditions. Current 
information does not make this possible; available measures are limited at best. Some 
relevant survey data are available, as indicated below; some of these surveys are conducted 
on an ad hoc basis only. Missing to a large extent is information anchored in the 
observation of science and mathematics classrooms that would augment and support 
findings from survey data (largely derived from self-reports of students, teachers, and 
school authorities). 

I. Standards for Curriculum and Learning Goals 

At the national level, standards for curriculum and learning goals should be 
developed. The standards and goals should delineate essential content knowledge 
matched with the skills and attitudes that constitute mathematical, scientific, and 
technological literacy and at the same time provide the foundation for further 
work in these fields. The Curriculum and Evaluation Standards for School 
Mathematics produced by the National Council of Teachers of Mathematics 
(NCTM) and Science for All Americans produced by the American Association for 
the Advancement of Science (AAAS) are exemplars of widely endorsed national 
standard-setting efforts. The existence of such standards does not suffice, 
however. 

To make national standards effective, relevant staff at every level will need 
to be aware of, adopt, and use these standards. The levels include State (central 
education staff), district (central education staff), school (principals and 
department chairs or equivalent), and classroom (teachers). Although awareness 
and adoption can be determined through surveys, use can only be established with 
any certainty through classroom observation. 

a. Available for the 1991 Progress Report. Very little is currently available, 
except for some information on relevant instructional practice in IEA, NAEP, and 
NELS:88 (see below). 
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b. To be developed for future Progress Reports 

(1) Development of data sources. Data sources that will provide 
information to set standards for curriculum and learning goals must be 
developed for the future. Information must be gathered on awareness and 
adoption of curriculum and learning goals and on use of the standards. To 
measure awareness and adoption, we recommend building appropriate 
questions into existing surveys (in order of preference): National Survey of 
Science and Mathematics Education (NSSME), to be repeated every 3 years 
(it is now ad hoc); Schools and Staffing Surveys (SASS) questionnaires 
(school and teacher level); Education Commission of the States' (ECS) 
surveys of reform practices and the Council of Chief State School Officers 
(CCSSO) indicator project State level; and NAEP teacher and school 
questionnaires. 

(2) Standards and goals. On use of standards and goals, see below, 
under instructional context, for appropriate proxy questions in NSSME, 
SASS, NAEP, and IEA. IEA would provide some useful data for 
international comparisons. Also needed are in-depth case studies linked to 
the survey questions. 

At the State level, the question translates into existence of State 
curriculum frameworks that incorporate the national standards. The same 
typology follows as at the national level: awareness, adoption, and use at 
the district, school, and classroom levels. No data exist currently, except 
possibly for the 1990 NAEP State-by-State mathcmaticsasscssment, which 
may have very limited information on instructional practice. In the future, 
the State-level surveys suggested above (ECS, CCSSO) could be used to 
collect information on existence of reform curriculum framework" in the 
States; awareness, adoption, and use could be tracked in the States with the 
measures recommended for the national level, provided the survey designs 
include State-representative samples. 

2, Instructional Conditions 

In general, instructional conditions should supprt the: learning goals and 
curriculum standards delineated at the national level. Tracking instructional 
conditions rueans giving attention to the following factors and their distribution 
across different population groups: 

School organization and structure: press for high achievement in mathematics 
and science for all students, courses offered (secondary school) or time devoted 
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to mathematics and to science (elementary school), organization of instructional 
time (e.g., flexible scheduling), and organization of teachers 1 rime and structure 
of decisionmaking. 

• Materials related to the curriculum: textbooks and associated materials, 
equipment and laboratory facilities, and availability of computers and 
telecommunications technology. 

• Instructional practices in the classroom: what is taught and how it is taught. 

a. Data available for 1991 Progress Report Current data sources on 
instructional conditions include NSSME, NAEP, NELS:88, and IEA. NSSME 
(data collected in 1986) and NAEP (1990 for science and math) have data on time 
spent, course offerings and mathematics textbook usage, and science and 
mathematics classroom activities. Hie NELS:88 original 1988 survey has some 
data; the 1990 followup survey has special supplements on the practices of 
science/mathcmaticsteachers of 10th graders. IEA has information on curriculum 
content coverage and provides international comparisons, but this information 
dates back to early- and mid-1980s. 

b. Data to Be developed for future Progress Reports. To develop appropriate 
data sources for the future, we recommend continuing to include key questions on 
the three sets of factors in periodic surveys like IEA, NAEP, and SASS, and 
regularizing NSSME. Much developmental work needs to be done to create an 
efficient set of tracer variables. At the same time, survey information must be 
augmented by intermediate-level and in-depth information. Intermediate-level 
information would consist of material like teachers 1 logs and time-use budgets, 
analysis of materials used in the classroom, and analysis of student work samples. 
In-depth information entails classroom observation linked to the survey and 
intermediate-level information. 

State-level information on instructional conditions should parallel the 
national level information. For survey information, the same sources can be used, 
provided they yield State-level as well as national information. (This is currently 
not the case for NSSME or NAEP, unless NAEP continues State assessments.) 
There are three unresolved issues at the state level: 

' Developing appropriate questions that get at key tracer (proxy) variables and 
reporting clear, concise, and not misleading information that might lead to 
simplistic and ineffective or counterproductive policy "fixes." 
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• Increasing sample sizes to make information state-representative would greatly 
increase survey costs and response burdens; currently there is much more survey 
information collected than has been adequately analyzed; and 

* Collecting in-depth information for every State through case studies or the like 
is not feasible; the intermediate-level information may have to suffice for 
reporting at the State level. 

3, Tests and Assessments 

At every level, tests and assessments must reflect the nationally defined learning 
goals and standards. At the national level, we agree with the Goal 3 report that 
NAEP data should be used for the present. The Goal 3 report provides reasons 
for the inappropriateness of other national-level tests. For the future, a major 
development effort is needed to bring NAEP mathematics and science assessments 
in line with the learning goals and standards enunciated by NCTM, AAAS, and 
other professional bodies. 

At the State level, two pieces of information should be collected: the 
policies on State and district testing, and information on the nature of the 
assessment or test being used. Policies on State and district testing include 
subjects, grades, frequency, sample or census design. Some of this information is 
currently available for States (not for districts) from ECS and CCSSO, but 
collecting it needs to be regularized through ongoing surveys. 

More important is the nature of the assessment or test being used at the 
State, district, and classroom levels. Current norm-referenced, commercially 
available tests are inappropriate because they do not reflect forward-looking 
learning goals and standards in science and mathematics education. 

Studies have documented that teachers also use very limited tests and have 
little training in the range of techniques they should be using to assess their 
students' progress in mathematics and science. Therefore, reporting on the kinds of 
tests and assessments used at the State, district, and classroom levels should be 
accompanied by sample tests and protocols for other assessment techniques (e.g., 
observation of individual student and group performance, student work samples, 
teacher logs, and sustained science projects). 

Two issues must be resolved for tests and assessments of instructional 
conditions to be effective: availability of test and assessment tools, and teacher 
awareness. 

First, there are not enough good test items and assessment exercises 
available to construct satisfactory large-scale (State-level and district-level) 
assessments, particularly in science. Current testing formats militate against 
assessing students' ability in designing and carrying out bona fide science 
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investigations. If testing is not to undo curricular and instructional reform efforts, 
a significant development effort is needed to align large-scale assessments with 
learning goals and standards in science and mathematics. 

Second, teachers need to learn, understand, and use assessment techniques 
appropriate to the learning goals and standards they are charged with achieving. 
Such teacher awareness will not be developed without a significant amount of 
relevant training incorporated in both prc-service and in-service education of 
teachers. A related need is understanding by school administrators (principals, 
district staff, State staff) of what constitutes a good test or assessment and how to 
report results to policymakers and the public. 

4. Provision and Allocation of Resources 

It is currently impossible to trace investments in science and mathematics 
education at any level — national, State, district, school, or classroom. The new 
Federal interagency coordinating effort should address this problem at the national 
level. 

At the State and district levels, budgets should be constructed 
programmatically so that resource investments can be tracked. Again, the 
information should be organized so that it can be disaggregated by population 
group. (For example, do districts/schools that serve largely poor students or have 
a black or Hispanic majority invest at equivalent levels compared to schools with 
a white majority or schools serving middle-class communities?) It would also be 
of interest to know the discretion provided to principals and teachers at the school 
and classroom levels to buy expendable equipment and materials for science. 

Another aspect of resource investment is the contribution to science and 
mathematics education from nonschool sources such as business and industry, 
professional societies, and community organizations. There is much anecdotal 
evidence of contributions of time, equipment, and other resources through 
partnerships and alliances with schools, but no systematic accounting exists. Such 
an accounting would serve two purposes, establishing the extent and kind of 
current involvement and encouraging its growth. 

Unresolved issues concern the difficulties of constructing program budgets 
(especially at the elementary level) and tracking investments by nonschool 
organizations. However, without having that information, a key piece of data will 
be missing that could account for progress or lack of it on the other components 
critical in "strengthening mathematics and science education throughout the 
system." 
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5. Attitudes and Expectations 

It is increasingly clear that part of America's student achievement problem stems 
from deep-seated beliefs and cultur dispositions that are starkly different from 
those in other countries. For studeni ^Jiievement to become first in the world, 
critical attitudes and beliefs must shift as well. Americans tend to believe that 
innate ability and inborn talent determine achievement in science and 
mathematics. Moreover, Americans, including educators at all levels, believe 
there are natural and inevitable tradeoffs between educational excellence and 
educational equity. However, Everybody Counts, the National Research Council's 
report to the Nation on the future of mathematics education, argues that excellence 
and equity excellence are compatible goals, converging on the single focal point 
of heightened expectations. 

It is proposed that, along with international, national and State student 
achievement trends, data be gathered and reported on the perceptions among 
mathematicsand science educators regarding the long- and short-term value of 
mathematics and science, the personal and global importance of mathematics and 
science, the reasons for mathematical and scientific achievements, the likelihood 
of an individual's success in mathematics and science, the differential expectations 
along racial or gender lines in mathematics and science achievement, and 
mathematics and science as disciplines. 

a. Attitudes on science and mathematics learning. Being first in science and 
mathematics learning will require a major shift in attitudes among students, 
teachers, and parents. America's student learning problems in science and 
mathematics stem in part from deep-seated beliefs that are different from those in 
other countries. Critical student, parent, teacher, and policymaker attitudes 
include: 

student self-perception with regard to learning potential/success; 

reasons for differences among students, including ethnic and gender 
differences; 

value/importance of science and mathematics on personal/global bases; and 
views regarding various disciplines in mathematicsand science. 
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b. International data. DEA mathematics and science studies include some 
student attitude data and could be used in the 1991 Progress Report. 

The International Assessment of Education Progress, with about 20 different 
countries participating, will be available in March of 1992 for the 1992 Progress 
Report. 

The IEA Third International Mathematics and Science Survey fHMSS — 
Third International Mathematics and Science Assessment) is planned, in two 
phases, for 1994 and 1998. Student attitudes toward mathematics and science will 
be investigated. Results will not be available until 1995 and 1999, respectively. 

c. State and national data for the 1991 Progress Report 

The National Assessment of Educational Progress (NAEP) The NAEP 
State (eighth grade) and national 1990 mathematics assessment includes 
items regarding attitudes toward mathematics learning. The results will be 
available for the 1991 Progress Report. The NAEP attitude questions were 
asked of students in the 1986 science assessment, and the results could be 
used for the 1991 Progress Report. 

The National Education Longitudinal Stiiriy --NELS:88 . The 1988 base 
year survey of eighth-grade students includes questions about student 
attitudes toward mathematics and science classes, and perceptions of their 
teachers. These are avc lable for the 1991 Progress Report. 

The National Survey of Mathematics and Science Education—NSSME . 
The 1985-86 NSMME included a few teacher/principal attitude items 
about science and mathematics. Results of this survey could be used in 
the 1991 Report Card. 

The I/ingitudinal Study of American Youth—IiSAY The LSAY has 
some data on student and parent attitudes. Data have been reported on 9th 
and 12th grades for a cohort that is being followed. 

d. Potential data available for future Progress Reports. The current plan is 
for NAEP 1990, 1994, and 1998 Science Assessments to include national data on 
student attitudes, similar to the mathematics assessment. The results of the 1990 
Assessment will be available for the 1992 Progress Report on a national basis only. 
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The NAEP 1992 mathematics assessment with attitude items will be ready for 
State (fourth and eighth grades) and national indicators in time for the 1993 
Progress Rc t --rt. 

The NELS:88 first followup survey of 10th graders conducted in 1990 will have 
some attitude data ready for the 1992 Progress Report. The survey of 12th graders 
will be conducted in 1992 with data available for the 1993 or 1994 Progress 
Report. 

The second National Survey of Science and Mathematics Education is under 
review by NSF. If funded, teacher and administrator attitude items could be 
included. It is likely that this report will be available for the 1993 Progress Report. 

e. Recommendations. The international comparison data arc likely to be limited to 
the three studies reported above — IAEP and TIMSS Phase I and Phase II. Reports 
should be available for 1992, 1995, and 1999. A strong recommendation is that the 
attitude items included in the NAEP be at least as strong as or parallel to items 
used in TIMSS. 

NAEP should be expanded to include additional student items on attitudes — 
particularly as they related to various disciplines within the sciences or content 
areas within mathematics. State data should be encouraged at all three levels of 
reporting in both mathematics and science. 

Teacher and principal items specific to mathematics and science education 
should be included as supplements to the Schools and Staffing Survey (SASS) — 
only weak attitude-proxy measures are available through the sample of science and 
mathematics teachers. 

Alternatively, the National Survey of Science and Mathematics Education should 
be conducted more regularly. Currently the study is funded on an ad hoc basis. 

C. Increasing the number of teachers trained in mathematics and science 

Objective 2: The number of teachers with a substantive background in 
mathematics and science will increase by 50 percent. 

This objective needs to be treated in two parts, pre-scrvicc and in-service 
education. Pre-service education encompasses the characteristics of undergraduavc 
and graduate education of prospective teachers. In-service education includes the 
characteristics and professional development of active teachers. 

Pre-service elementary teachers frequently study little or no mathematics and 
science beyond high school. Their post -secondary coursework preparation for 
teaching these courses often consists of a one semester "methods" course. These 
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coursework alone will not be sufficient; prospective teachers must also team how to 
engage students in mathematical and scientific problem solving in ways that are 
consistent with the national standards. The pre -service education of secondary 
mathematics and science teachers must also ensure that these teachers are equipped 
to engage students in ways consisted with the national standards. We must develop 
ways to measure qualitative aspects of pre-service teacher education. 

Assessing the substantive background in mathematics and science of teachers 
currently in the classroom has two aspects: tracking relevant characteristics of the 
teachers and tracking the amount and quality of staff-development opportunities 
available to them. 

i. Tracking Teacher Background Characteristics 

To track relevant teacher characteristics, profiles of the current mathematics and 
science teaching force must be assembled. The profiles must include elementary 
and middle-school teachers and should gather data on training, number and types 
of degree, age, experience, and so on, of the teaching force. This information 
should be collected by teaching assignment and should also be available by student 
population group. The relevant surveys should include State-representative samples 
so that the information can be reported both nationally and for each State. 

There arc several current data sources for characteristics of high school teachers: 
NELS:88 and the HS&B teacher followup ATS. These are one-time samples, 
however, and will be of little help beyond the 1991 Progress Report. SASS collects 
some of the relevant information as well. For the future, a special 
sciencc/mathematicssupplement to SASS should furnish the relevant information. 
Alternatively, NSSME should be regularized, conducted every 3 years, and used to 
collect this information. 

Teacher characteristics surveys should also include the proportions of current 
mathematics and science teaching force certified by current State license 
requirements, traditional and alternative license procedures, and temporary and 
emergency certificates. Such a survey will yield noncomparable information across 
States, especially at the elementary and middle-school levels. The importance of 
these data rests in distributional questions, for example, who does and who docs not 
get certified teachers. SASS, CCSSO, and NSSME have pertinent information for 
1991; they could continue to collect it in the future. A useful addition would be 
logging the States that have incorporated certification standards recommended by 
such national bodies as the NCTM and the NSTA. 

To complete the teacher characteristics, data should be collected on the 
percentage of teachers with national board certification in science or mathematics. 
These data arc available from the National Board for Professional Teaching 
Standards. 
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2, Tracking Teacher Development Opportunities 

Assessing mathematicsand science teachers' background must also include 
surveying the availability, quantity, and quality of staff development opportunities 
for teachers at every level from elementary through secondary. Determination of 
quality should include measures of training in the use of computer and 
telecommunicati ns technology in science and mathematics teaching, as well as 
training in assessment and testing strategies that match learning and auricular goals 
in science and mathematics. Hie tracking of opportunities should include those 
formal and informal staff development opportunities offered outside the regular 
district, State, and university structures (for example, by private-sector businesses 
and organizations). Both SASS and NSSME have some information; they could be 
expanded to collect more. Information on district and State policies with respect to 
staff development should also be collected. 

3. Unresolved Issues 

A number of issues concerning the assessment of science and mathematics teachers' 
backgrounds remain unresolved. We determined that teachers possessing an 
adequate science and mathematics background are a necessary but not sufficient 
prerequisite to teaching science and math. Teachers must also be able to construct 
teaching situations in which students will learn mathematics and science. In part, 
their competence to do so can be assessed through tracking teaching practices as 
recommended under the instructional conditions section of Objective 1. Teacher 
background assessments are important to consider under the second objective as 
well because of the need to build teaching competence in mathematics and science 
into pre-service and in-service education and to track the extent to which this 
objective is met. 

Assessments of teachers' science and mathematics background are particularly 
difficult to achieve for elementary school teachers. Even though NSTA and NCTM 
have formulated standards for science and mathematics preparation for elementary 
school teachers, these hardly constitute a "substantive" background. Moreover, 
most teachers in the schools now did not receive preparation matching the 
standards. The relevant courses are not available on many campuses, either for 
prospective teachers or for in-service training. Note that much of the language 
throughout discussions of teachers in mathematicsand science refers to mathematics 
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and science teachers, which completely ignores elementary school teachers and 
many middle-school teachers. 

The second objective is silent on the need for principals and other administrators 
to be sufficiently prepared to understand what the teachers need to know and do. 
Some thought must be given to how to assess administrators' support and 
constructive criticism for science and mathematics teachers and how to collect the 
requisite information. SASS may have some relevant information and would 
provide a vehicle for the future. 

4. General Recommendations 

To achieve Objective 2, the Resource Group made some general recommendations: 

• A focused survey or a set of coordinated surveys should be constructed to 
collect regularly the information needed to track Objectives 1 and 2. At present 
we have focused surveys that are not conducted regularly. We also have 
broad-purpose surveys that are conducted regularly but do not collect all of the 
necessary information. The lack of an integrated plan for data collections makes 
it difficult to assemble the appropriate data for the Progress Report. 

• Survey information is broad, but not deep. It must be augmented by in-depth 
probes that reveal the realities of the classroom. An appropriate model is the 
combination of studies supported by NSF in 1976-77, including a survey of 
science and mathematics education (the forerunner of NSSME) together with a 
set of case studies of classroom practice involving classroom observation, 
interviews, and local-level surveys. 

For some purposes, intermediate-level information should be collected on 
a sample basis. Examples include but are not limited to identifying curricular 
content and goals at the classroom and school levels as described above, 
teaching practices in science and mathematics classrooms, and content of science 
and mathematics courses for prospective elementary school teachers. 
Appropriate techniques arc being developed by the Center for Research on 
Evaluation, Standards and Student Testing at the University of California at Los 
Angeles. 

• Research and development must be supported in two areas: the effective 
incorporation of computer and telecommunication technology in science and 
mathematics education, and the creation of assessment exercises and strategics 
consonant with the nationally enunciated mathematics and science learning goals 
and standards. 
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D. Enrollments and the Science and Mathematics Pipleline 

Objective 3: The number of VS. undergraduate and graduate students, 
especially women and minorities, who complete degrees in mathematics, 
science, and engineering win increase significantly. 

J. Elementary and Secondary Education 

a. Measures for 1991 Progress Report International figures on enrollment and 
participation at the K-12 level are problematical. The structures of nations' school 
systems vary fundamentally in terms of the kinds of programs in which students 
participate at various grade levels. Specifically, beyond the age of 13 or 14, 
fundamental differences exist in the structures of public-private, vocational- 
general, and selective-open programs of various nations. Simply defining 
"enrollment" at a basic level in comparable terms is difficult. Beyond these 
problems, defining a "course" in mathematics or science is difficult. For instance, 
the U.S. uses a "layer-cake" approach in both areas, wherein students proceed 
through a sequence of courses each year in geometry, algebra, and calculus, or 
history, chemistry, and physics. In other countries, algebra or physics may be 
studied each year along with other subjects. 

This has led to comparing the content students study by describing the topics 
they cover, regardless of course or program structures. Internationally, we should 
summarize for 1991 results of comparative coverage of curriculum from recent IEA 
studies in mathematics and science. These indicate in general that the U.S. 
curriculum in mathematicsis relatively repetitive and unambitious. It is strongly 
urged that international studies in mathematicsand science conducted during the 
1990s include data on curricular coverage among students in participating countries 
and that this include measures that allow monitoring trends in coverage, preferably 
back to the studies completed in the 1980s. The 1991 report should include 
relatively brief, summarative data from the most recent studies on the relative 
extent to which U.S, students are exposed to a range of curricular topics in 
mathematicsand science. 

Nationally, two sources are available for data on enrollment in mathematicsand 
science courses. As will be developed below, these figures are limited in what they 
tell us about the course content students receive, and they are subject to abuse or 
manipulation if they are used as high-stakes indicators. However, along with other 
date, they can be useful. 

National data have been obtained from studies of samples of students' 
transcripts. These studies are available for students who were in the National 
Assessment of Educational Progress (NAEP) sample in 1987-88 and for high 
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school seniors in 1982. Further, a study is planned for 17-year-olds tested by 
NAEP in 1989-90. This information should allow some analysis of trend over time. 

Hie other source of course-enrollment data is the States. More than forty states 
have enrollment data using comparable definitions and procedures. This could 
allow aggregation of national estimates or at least "media state" figures. 

Enrollment rates (percent of students who take a course sometime in their high 
school career) should be reported from the existing data in September, 1991. Data 
should be reported by a standard taxonomy of course content and should include 
break-cuts by gender, ethnicity, and socioeconomic status, if available. 

Course enrollments by themselves may mask actual course content. For 
example, what is included in a course titled "Algebra 1" may vary considerably. 
Further, reporting our status on course enrollments may stimulate spurious increases 
in enrollments without concomitant improvements in instructional content Finally, 
attention to these data as a sole criterion of instructional content might further 
institutionalize course structures we do not wish to perpetuate (dividing "algebra" 
into two courses, instead of providing an integrated mathematics course as 
recommended by the national standards). 

For these reasons, other, more detailed information on course content should be 
included. Data on topic coverage and instructional approaches are available 
nationally from NAEP, SASS, and NELS. These should be used in 1991 to 
augment and amplify course-taking figures. 

State-by-State, course-taking data in mathematicsand science are available from 
CCSSO for about 40 states. This information is broken down by standard course 
titles and categories in each field and includes figures by gender for many states. 
Enrollment by race-ethnicity is available for few states. 

By Fall 1991, data will be available from NAEP and SASS State-by-State at a 
relatively general level on relative topic coverage and instructional approach. For 
example, NAEP asks teachers the extent to which they emphasize several (less than 
a dozen) sub-areas of mathematics. This can be summarized State-by-State. 

b. Measures for Future Progress Reports, International data available bv the year 
2000 will allow substantial enhancements of the kinds of data that can be reported. 
Planned international math-science studies in 1994 and 1998 should monitor trends 
in curriculum coverage over time, tracking back to the mid-1980s studies. Topic 
coverage or "opportunity to learn" variables are being built into these studies. 

Natio nal and Statc-by-Statc data for the year 2000 should be fundamentally 
shifted. Course-taking data should be institutionalized at the state and national 
levels using state data systems and transcript studies. Especially important is 
development of data by gender and race/ethnicity. However, the United States has 
accurate, descriptive information on what students are taught. Ultimately, this is 
more appropriate than course-taking. We should begin regular surveys of 
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curriculum coverage at the national and state levels in math, science, and other 
subjects. This could be done in conjunction with NAEP or SASS or through a new 
data-collection program. 

2. Postsccondary Education 

America's colleges and universities have a vital role to play in making America first 
in the world in mathematicsand science. Some databases are already available for 
progress reports on college enrollments and degree production in U.S. colleges and 
universities. These databases include race, sex, and citizenship distributions. But 
much work needs to be undertaken for the important international comparisons and 
for developing appropriate ways to measure the quality of college mathematicsand 
science curricula and student achievement. 

Higher education's role can be viewed as twofold: First, to improve 
mathematicsand science instruction and outcomes in the primary and secondary 
schools by producing more mathematicsand science teachers; second, to strengthen 
the undergraduate mathematics and science curricula for college students majoring 
in mathematicsand science disciplines, for students preparing to become teachers, 
and for students who are not majoring in mathematicsand science but are pursuing 
associate and baccalaureate degrees in other disciplines. 

The first of these two roles, training more teachers, is explicit in the objectives 
of Goal 4. The latter role is vitally important, but it is at most implicit in the 
objectives of Goal 4 because most people interpret the objective of strengthening 
curricula to apply only to elementary and secondary schools. Strengthening 
curricula in college-level mathematicsand science may be overlooked because the 
common perception is that America has superior colleges and universities, but 
inferior primary and secondary schools. That perception is influenced primarily by 
the strength of graduate programs in VS. universities, which are attracting an 
increasing number of students from abroad, and because of the great research 
productivity of the Nation's leading university researchers. That popular perception, 
however, does not stem from the numbers of American citizens receiving 
mathematicsand science bachelor's, master's, and doctoral degrees or from the 
strength of undergraduate mathematicsand science curricula. The number of 
Americans majoring in mathematicsand science is declining, and there are no 
national or international assessments of mathematicsand science education that 
compare the knowledge and skills of recipients of U.S. baccalaureate degrees with 
their counterparts abroad. 

Cooperative arrangements need to be established with other nations to provide 
comparable productivity indices. There is no available evidence that shows whether 
the undergraduate science and mathematicscurricula in America's college and 
universities are internationally competitive or whether the American recipients of 
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baccalaureate degrees regardless of major have acquired Che matbematicsaud science 
skills that make them internationally competitive. Hie tact that American colleges 
and universities attempt to educate the masses rather than a select few, as in most 
other nations, suggests that the goal of becoming first in undergraduate mathematics 
and science education is no less challenging than it is at the primary and secondary 
school levels. 

a. Monitoring Progress. The data and information for the higher education 
progress report should correspond to objectives and contributions that higher 
education attempts to make toward the goal. The objectives of Goal 4 that pertain 
directly to higher education are 

• The number of teachers with a substantive background in mathematics and 
science must increase by 50 percent. 

• The number of U.S. undergraduates and graduate students, especially women 
and minorities, who complete degrees in mathematics, science, and engineering 
must increase significantly. 

b. Data Available far the 1991 Progress Report. The Integrated Postsecondary 
Education Data System (IPEDS) is an annual survey conducted by the U.S. 
Department of Education that provides baseline 1991 data on the number of 
recipients of baccalaureate, master's, and doctoral degrees by major field, 
ethnicity, and sex in United States colleges and universities. Institutional 
comparisons within States are also possible. 

This database can also be used to monitor progress through the year 2000 
and beyond. Colleges and universities participate in this survey voluntarily, but 
the overwhelming majority participate. This data base, however, does not show 
the number of teacher education majors who have an emphasis or concentration in 
mathematicsor science. 

The IPEDS does not provide data on the number of Americans or foreign 
nationals who enroll or receive their degrees abroad, but it does report the number 
of foreign nationals enrolled and receiving degrees in participating U.S. colleges 
and universities. For the progress report to be complete, the degrees awarded by 
colleges and universities in Canada, Europe, Japan, China, and so on, in 
mathematics and science fields and in mathematicsand science teacher education 
disciplines comparable to those offered in U.S. colleges and universities must be 
monitored. 

The Division of Science Resources Studies of the National Science 
Foundation (NSF) annually conducts the Survey of Graduate Students and Post 
Doctorates (SGSPD) in science and engineering fields. The survey involves all 
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departments in science disciplines at all U.S. colleges and universities that offer 
master's and doctoral degree programs in the sciences, including all 
environmental, mathematical, computer, agricultural, biological, social, 
engineering, and health fields, as well as psychology. The results from this 
survey can be used to monitor trends in student enrollment hi U.S. master's and 
doctoral degree programs including the sex, race/ethnicity, and citizenship of 
students enrolled. This survey also reports the aggregate financial assistance data 
that students receive at these graduate schools. Like IPEDS, however, it does not 
include an international component. 

Adequate measures for judging the strength and quality of college and 
university mathematicsand science curricula or for international comparison do not 
exist for the September 1991 Progress Report. Such measures need to be 
developed for the year 2000 and beyond. The science and mathematics parts of 
the GRE are inappropriate for this purpose because they were designed for and are 
taken only by the select subpopuiation of recipients of baccalaureate degrees who 
plan to attend graduate school; therefore, the context of these tests does not reflect 
the achievement outcomes expected of the general population of college graduates 
or the general population of science and mathematicsmajors in undergraduate 
school. In other words, the results of these tests do not represent the overall 
strength of the undergraduate curriculum or the science and mathematicsmajor 
curricula. 

The Major Field Achievement Tests (MFAT), developed jointly by the 
Educational Testing Service (ETS) and Graduate Record Examination Board 
(GREB) in 1989, are multiple-choice tests used by a small number of colleges 
and universities to assess the achievement of college seniors in mathematicsand 
science (biology, chemistry, and physics) curricula. Some of the science 
professional societies have also developed standardized multiple-choice tests for 
college seniors majoring in the respective science disciplines. While these may be 
useful instruments to build on, they are not sufficient for assessing progress 
toward achieving national goals or for international comparison for the following 
reasons: 

• They do not reflect national consensus among educators on the skills and 
knowledge that mathematicsand science majors graduating from U.S. colleges 
and universities should acquire. 

• They do not reflect a national consensus on the science and mathematics 
knowledge and skills that graduating college seniors who are not science and 
mathematicsmajors should acquire. 
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They do not reflect a national consensus on the methods of assessment that 

3S SSS fecul * ^ *- * uscd * d --C 



• In their present form they do not lend themselves to international comparisons 
nor do they reflect an international agreement about the sdencHnT 
^T^Jf 1 r W ' ed8e «» ""^ S^duates In mathematicsand 
£3 C$ J** W ^ *"* «*» "V * — ^ 

c. Recommendations for Future Progress Reports. We recommend that: 

' SK^sr"* annual,y ,he ,rends on degrees confeired * »* 

The SGSPD be used to report and monitor trends in graduate student enrollment 

~o,htXi,oS^. mabenia,iC8 - aSlith 

The President and the Governors mandate an examination system that oennits 
^ » "teasurn both the strength of JS^SSL 

and mathemat.es curncula and the skills and achievement levelfof studenu 
gradual mg from science and mathematics curricula in VS. colleges and 
un.vers.ties compared with other developed nations. This assessWwill 
requ.re mternational committees with representatives from prtS ^nations 
to plan and develop these examinations. ^ 8 n 

IV. Technology and Science and Mathematics Education 

The i impact that technology has had on the whole of society is not evident if one looks 
only a, Amencan education. Technology has revolutionized the pmcS yt 
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science, and engineering; yet, it is little used for mathematics and science education. Only 
a few select schools are equipped with sufficient laboratory equipment and computing and 
telecommunications technology. Distance-learning techniques will become important tools 
for mathematics and science education. The exploitation of technology for improving 
education systems must be accelerated. This endeavor will challenge our imaginations and 
strain our budgets. 

Vision and leadership at the highest levels of government are necessary. We must 
also recognize that large amounts of Federal support will be required to develop the 
educational technology required to support an advanced mathematics and science education 
system throughout the Nation. Individual schools, districts, or States cannot meet the 
financial requirements of advanced interactive systems. Federal support will also be 
required to encourage and ensure an adequate supply of well-prepared teachers who are 
able to use this technology and to address the new curriculum goals these initiatives 
require. 

The Resource Group strongly supports the integrated interagency coordination 
started by the Committee on Education and Human Resources (CEHR) within the Federal 
Coordinating Council for Science, Engineering and Technology (FCCSET) of the Office of 
Science and Technology Policy. The emphasis placed on precollege education by CEHR 
is strongly endorsed. The efforts of this committee as well as the Systemic Initiative 
supported by the National Science Foundation should have a major impact on activities 
related to Goal 4. 

The Resource Group supports the development of a national clearinghouse concept 
that would integrate existing information sources, provide rigorous quality control, interact 
with FCCSET- CEHR and other organizations, and serve as a national information resource 
for instructional materials, outstanding programs and activities, and assessments. 

We must develop a sound infrastructure on which to build the educational system 
that will equitably provide world-class mathematics and science education for our students. 
If wc do not create the necessary infrastructure, we will not realize our goal. 

V. Epilogue: A Vision of Science and Mathematics Education in the 
Year 2000 

The following is our vision of a first-class education system for in the year 2000. Guiding 
principles will include the following: 

• Each education system is guided by the basic principle that each student can and 
will attain mathematics numeracy and scientific literacy. 

• The standards of learning expected for each child when achieved will enable 
successful participation in future science, mathematics, and technical education 
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and work opportunities in a world where science, mathematics, and technology 
play a critical role. 

• The predictive power of sex, race, and ethnicity with regard to successful 
science, mathematics, and technical learning will be eliminated. 

• Our vision anticipates that in the year 2000, 

Teachers will possess the requisite mathematics, science, and technical 
knowledge and pedagogical skill and have the time to learn new 
content in science and mathematics, to gain an understanding of 
student learning and development, and to explore alternative ways to 
design active learning opportunities. 

Each student will be fluent in the language and culture of mathematics 
and attain scientific habits of mind. 

Students will experience a variety of active learning opportunities and 
teaching strategies designed to encourage participation and motivate 
continued study in mathematics, science, and technology. 

Students will be engaged in learning opportunities that are authentic 
with regard to mathematical, scientific, and technical work, including 
the use of technologically up-to-date equipment that is aligned with 
the tools of science (calculators, computers, rich databases, optical and 
electronic networks and telecommunications systems, energy sources, 
and measurement instruments). 

Students will enjoy the pursuit of knowledge driven by their natural 
curiosity and the construction of knowledge from the experiences and 
resources provided. 

Working in teams and individually, each student will be responsible 
for his or her own science and mathematics learning. 

Students will undertake science projects in which they will experience 
discovery by amassing evidence; making and checking hypotheses; 
searching, for and fmding patterns; theorizing; experimenting; and 
making conjectures, proving relationships, and applying them to real 
and hypothetical situations. 
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Each student will have the concentrated time, quality instruction, and 
multiple opportunities to learn, including second and third chances to 
make up for lost or missed content, courses, or requirements. 

Each student will be judged by authentic examinations and 
assessments of knowledge, understanding, and skill that reinforce 
mathematics and science learning goals and are integrated into the 
education program of the school. 

• Further, our vision imagines modern school systems in which 

Each h>:»\(H» vill be operated and organized on the principle that 
student lea ving in science and mathematics is among the highest 
priorities in the allocation of time and other resources, recruitment of 
personnel, design of assessment and accountability systems, and use of 
facilities and equipment. 

Each school will be structured so that people making decisions about 
the mathematics, science, and technological programs are 
knowledgeable in those areas and are guided by the active involvement 
of scientists, mathematicians, engineers, and persons with technical 
expertise. 

Schools will be governed so that those responsible for implementing 
learning programs are actively engaged in design teams that have the 
flexibility to deploy people, time, materials, equipment, and facilities 
in ways that lead to more productive and lasting learning of science 
and mathematics. 

Schools will be imbued with a spirit of experimentation in science and 
mathematics education that encourages creative, innovative approaches 
to learning with results documented and used to inform future 
decisions. 

Each school will be equipped with a rich variety of resources 
connected through a system of electronic telecommunications 
networks. These networks will access resources for teaching, learning, 
and professional development and will be able to communicate in a 
timely manner with each other, with important constituencies, and with 
those engaged in ironitoring the condition of the education system. 
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Schools will be committed to actively »e suiting high-quality teachers 
to ensure that new teachers are inducted and prepared to take on a 
variety of roles. 
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Chapter 5 
Adult Literacy and Lifelong Learning 

An Interim Report From the Resource Group on Adult literacy and 

Lifelong Learning 

GOAL 5: By the year 2000, every adult American will be literate and will possess the 
knowledge and skills necessary to compete in a global economy and exercise 
the rights and responsibUttes of citizenship. 

Objectives: 

• Every major American will be involved in strengthening the connection between 
education and work. 

• And workers will have the opportunity to acquire the knowledge and skills, from basic 
to highly technical, needed to adapt to emerging new technologies, work methods, and 
markets through public and private educational, vocational, workplaces, or other 
programs. 

- The number of high-quality programs, including those at libraries, that are designed 
to serve more effectively the needs of the growing number of part-time and mid- 
career student will increase substantially. 

• The proportion of those qualified student (especially minorities) who enter college, 
who complete at least two years, and who complete their degree programs will 
increase substantially. 

• The proportion of college graduates who demonstrate an advanced ability to think 
critically, communicate effectively, and solve problems will increase substantially. 
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In early 1991, a Resource Group on Adult Literacy and Lifelong Learning was convened to 
recommend indicators and strategies for measuring progress toward achieving this goal. 
Members of the group are as follows: 



Mark Musick 

Paul Barton 
Forest Chisman 
Peter Ewell 

Patsy J. Fulton 
James R. Morris, Jr. 

William Spring 
Tom Sticht 

Marc Tucker 



Southern Regional Education Board, Atlanta, Georgia 
(convener) 

Educational Testing Service, Princeton, New Jersey 

Southport Institute for Policy Analysis, Washington, D.C. 

National Center tor Higher Education Management Systems, 
Boulder, Colorado 

Oakland Community College, Bioomfield Hills, Michigan 

South Carolina State Board for Technical & Comprehensive 
Education, Columbia, South Carolina 

Federal Reserve Bank of Boston, Boston, Massachusetts 

Applied, Behavioral, and Cognitive Sciences, Inc., El Cajon, 
California 

National Center on Education and the Economy, Rochester, New 
York 



The Panel and Goal 5 Resource Group welcome your reactions to the Interim Report on Adult 
Literacy and Lifelong Learning. 
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National Education Goals 
Goal Five 

Adult Literacy and Lifelong Learning 



L Introduction 

Measuring the Nation's and States' progress toward achieving the national education goal for 
adult literacy and lifelong learning poses a special challenge. The challenge begins with the 
fact that there is no single definition of literacy. This goal and its objective encompasses 
multiple definitions, ranging from the most basic literacy skills through workforce skills 
demanded by an internationally competitive economy and baccalaureate-level skills of 
well-prepared college graduates. Consequently, this goal and its objectives demand a wide 
range of measures. Only by including indicators of the extent to which the Nation and States 
are making progress on the objectives as well as the larger goal can we obtain a full picture 
of adult literacy and lifelong learning. 

The goal is also difficult to measure because there is no "system" that is being 
measured. The literacy and workforce programs referred to are scattered throughout 
society — at the workplace; in unions; in employment and training programs; in libraries, 
YMCAs, and myriad voluntary and assocational programs; and in public and private 
vocational-technical schools, colleges, arid universities. The State role may include providing 
incentives to expand opportunities, to fill gaps in services to adult learners, and to ensure the 
quality of service through accreditation and licensing if public funding is provided. To 
collect the data in order to report on State and national progress in 1991 or to develop an 
effective data system, we will need to rely on the cooperation and good will of these varied 
providers and individual citizens. 

Beyond the challenge of reporting on a vast array of private-sector activities, public 
entities will need a more systematic means of reporting information on the participation and 
achievement of their participants. This is particularly true for the postsecondary level. Many 
States now lack the ability to track students through the postsecondary system, although some 
powerful State models do exist. All States lack the ability to report systematically on the 
ability of their college graduaics to think critically, communicate effectively, and solve 
problems. 
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n. The September 1991 Progress Report 



A. Data Reported for the Nation as a Whole 

i. Recommended Measures 

We recommend inclusion of national indicators from the following eight national data 
sources' baseline measures for the September 1991 Progress Report. We will also note 
the ability of these sources to provide State-by-State data. 

For baseline measures of the goal of literacy, the Resource Group recommends the 
following: 

a. Scores from the 1985 National Assessment of Educational Progress (NAEP) young 
adult literacy assessment No comprehensive national information is currently available 
on adult literacy. Baseline information will become available from the National Adult 
Literacy Survey in 1993. For September 1991, we recommend data from the 1985 
National Assessment of Educational Progress report on the literacy skills of young 
adults ages 21 to 25 years. The survey assesses three types of literacy skills: prose 
comprehension (measuring the knowledge and skills needed to comprehend texts); 
document literacy (measuring ability to use and interpret practical documents, such as 
bus schedules and job application forms); and quantitative literacy (measuring ability to 
apply basic arithmetic in everyday uses, such as checkbooks or order forms). These 
same categories of skills are assessed in the Department of Labor's literacy assessment 
of special populations and the National Adult literacy Survey that is currently under 
way. Thus, the NAEP young adult survey provides a partial baseline for future studies 
and longitudinal analysis. The information can also be reported by racial/ethnic group 
and by level of educational attainment. 1 

b. The Department of Labor assessment of literacy skills of special populations of 
under-employed or unemployed workers. The Department of Labor (DOL) 
commissioned the Educational Testing Service to conduct an assessment of Job 
Training Partnership Act participants and applicants for employment service and 
unemployment insurance applicants. The survey results can be analyzed according to 
racial/ethnic classification, educational attainment, and income status. The assessments 
were conducted in 1989 and 1990, are currently being analyzed, and should be 
available for the September 1991 Progress Report. Two States — Mississippi and 
Oregon—have also modeled their own assessments of adult literacy on the DOL 
survey; data from these surveys will be available for the 1991 Progress Report. 
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The Department of Labor does not plan to continue these special assessments. 
The Resource Group recommends that the Panel weigh the usefulness of these reports. 
If they are found useful, the Panel may want to recommend that DOL repeat these 
assessments later in the decade. 

c. Armed Services Vocational Aptitude Battery. The Armed Services Vocational 
Aptitude Battery (ASVAB), which includes the Armed Forces Qualification Test 
(AFQT), is a vocational aptitude assessment of the basic and technical literacy skills of 
individuals who are seeking to enter the workforce by applying for military service. It 
is taken only by persons seeking to be enlisted personnel, approximately 700,000 
persons per year. Data tor all of the tests are available by racial/ethnic group and 
gender. While this information represents a limited and self-selected segment of the 
population, it is an important and underutilized source of data about the basic skills and 
knowledge of a portion of American high school students and young adults. The 
AFQT consists of two reading tests and two math tests. The cutoff composite score on 
these tests is equated to approximately the 6th grade level and has been set by Congress 
as a score below which individuals are ineligible for military service. 

The Resource Group recommends that AFQT scores be reported on a national 
basis. We do noi recommend reporting scores on a State-by-State basis, since the 
variability among the population taking the test in different years within a State is too 
great. Wc also recommend that this assessment battery be renormed no later than 1995. 

For baseline measures of the objectives under Goal 5, the Resource Group 
recommends the following: 

d. Department of Labor survey on workforce preparation. DOL is conducting a 
survey of how workers receive their skills preparation. The Resource Group 
recommends using the results from this survey (if they are available) for the 1991 
Progress Report. This survey is a supplement to the Current Population Survey 
(January 1991). It may be possible to compare the results of the 1991 survey with a 
similar survey conducted in 1983 to show a trend, but there is a limitation: The 1983 
survey focused on training-related activities rather than educational services. If 
available, the data from this survey will be a useful indicator for establishing a baseline 
on Objective 2, but with a caveat. Objective 2 refers to workers* opportunities to 
acquire the basic to highly technical knowledge and skills needed in a competitive 
workforce. The caveat is that the survey will provide information on workers' 
participation in a range of education and training programs, but it will not provide 
information on the extent to which education and training opportunities are available to 
workers. New surveys of the business and educational communities will be necessary 
to complete the picture of existing opportunities. In addition, data may be reported for 
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specific States and/or large cities with larger samples in the survey, but they will not be 
available for all States. 

e. National Household Education Survey (NHES). The s? ce of the NHES is the 
National Center for Education Statistics (NCES). The Resource Group recommends 
that the Panel use the NHES survey to measure American participation in adult learning 
programs. This information will serve as a baseline for Objective 2, referred to above, 
and Objective 3, i.e., quality programs to serve part-time and mid-career students. 

Hie NHES is a new data collection system that is being implemented in the spring 
of 1991. The Resource Group recommends that the Panel ask NCES to provide 
information from the 1991 survey for the September 1991 report, including the 
percentage of adults enrolled in courses/training; the percentage of adults paying for 
their courses/training; the percentage of adults enrolled part time; and the percentage of 
adults enrolled in various categories of programs, such as programs for literacy, 
management, or specific job skills. While the survey will be conducted annually, only 
the 1991, 1994, and 1997 surveys will ask questions about participation in adult 
learning programs. Hie survey will provide national, but not State, data. 

As with the DOL survey of workforce preparation, the NHES will provide 
important information about participation in adult learning programs but will not 
provide information on the need adults perceive for such programs and the actual 
opportunities available to them. 

/ Integrated Postsecondary Education Data System (IPEDS) enrollment information. 
The source for IPEDS enrollment information is enrollment reports from vocational 
education, colleges and universities, and proprietary institutions. The Panel can use 
IPEDS as one measure of opportunity to acquire needed knowledge and skills. IPEDS 
collects information on all 11,687 postsecondary institutions and educational 
organizations, accredited and nonaccredited. The IPEDS system collects 
institution-level data on enrollments, program completions, faculty/staff, and financing. 
The Resource Group recommends that the Department of Education and the National 
Education Goals Panel staff develop a plan for presenting the enrollment information at 
both the national and State levels. 

g. Associate and baccalaureate degree recipients from longitudinal studies conducted 
by the National Center for Education Statistics: National Longitudinal Study of the 
High School Class of 1972 (NLS:72) and High School and Beyond which started in 
1980 (HS&B:8Q). These studies track the education and employment of students who 
completed high school in 1972 and 1980, respectively, and relate to Objective 4, which 
calls for substantial increases in college attendance and degree completion (associate 
and baccalaureate), particularly for minorities. Followup studies of these high school 
graduates provide a national indicator of college attendance patterns, including 
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persistence and completion. The class of 1972 can be compared with the class of 1980, 
and these trend data on attendance and graduation can be repotted in the September 
1991 Progress Report. The data should be repotted by racial/ethnic category. 

A third longitudinal study was begun in 1988 of the 8th grade class. Followup 
studies of that class in 1992 and later in the nineties will provide additional information 
about college attendance and completion patterns for these students. There is a State 
option to participate in the NLS, and a few States have done so. in a tew other States 
the numbers of individuals sampled are large enough that information on these States is 
available. These two combinations amount to 12 States where state longitudinal data 
could be reported tor the 1980 high school seniors. The Resource Group recommends 
that States participating in the longitudinal studies be asked to supply these data. Even 
limited State data may encourage more, and more useful, State data. In addition, there 
is an option available (NELS:88) for any State to contract for a sample large enough to 
yield State results. 

Given the three longitudinal studies described above, it will be possible to monitor 
progress through the decade using data from the 1991 report as the baseline. 

h. "Jse IPEDS enrollment and degree information to calculate a graduation rate. The 
Resource Group recommends that the Panel use a proxy measure as described below to 
report graduation rates at both the national and State levels until States' student 
unit-record systems, or their equivalent, are in place. Longitudinal studies have the 
disadvantage of being available only for a limited number of States. To deal with this 
problem in the short run, we suggest using the proxy measures from IPEDS. 

IPEDS collects information on first-year student enrollments, age of cnroilces, and 
number of associate and baccalaureate degrees awarded, by race and ethnicity. Until 
effective systems are in place to track student enrollment patterns within a higher 
education system or State, a proxy or indirect measure of graduation rates may be 
produced. The proxy is determined by dividing the total number of first-time students 
in a given year into the number of students completing associate degrees 4 years later 
and the number completing baccalaureate degrees 6 years later, or 150 percent of the 
expected time of completion to a degree. (One hundred fifty percent of expected time 
is also the figure being used in calculating graduation rates in the Students Right to 
Know legislation [see below]). The graduation rate calculated from IPEDS data would 
be an interim measure of Objective 4. 

i. State survey on number of high school graduates by racial/ethnic group. In order to 
report on the proportion of qualified students who enter college out of high school, the 
Resource Group recommends that the Panel request from each State the racial/ethnic 
data on its high school graduates. This information on high school graduates can be 
used in conjunction with information on college attendance and first-year enrollments 
from IPEDS to measure progress toward this objective indirectly. 
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2. Measures not Recommended for Publication in the September 1991 Report 
We recommend against using the following: 

Graduate Record Examination (GRE) scores. The Resource Group docs not 
recommend using the Graduate Record Examination taken by some college graduates as 
a measure of Objective 5, whi>Ji refers to college graduates' ability to think critically, 
communicate effectively, ana ^ve problems at an advanced level. The use of the 
GRE as an indicator has the same basic problems as the use of Scholastic Aptitude Test 
(SAT) and American College Test (ACT), referred to in the report of the Achievement 
Goal Resource Group. An unscientific and self-selected sample of students take the 
GRE Therefore, changes in the population of students taking the examination can 
effect the average scores in a given year more than what college graduates know 
and are able to do. While a majority of high school graduates take the SAT and ACT, 
a small minority of college graduates — namely those who are applying for selected 
graduate programs — take the GRE. 

3. Unresolved Issues 

There is no indicator available for the September 1991 report relating to Objective 1, 
strengthening the school-to-work transition. 

The Resource Group also has found that there is little information on opportunities 
for training. 



B. Data Reported for States 

1. Recommended Measures 

In addition to State data from IPEDS and from longitudinal studies, we are 
recommending that new surveys be administered immediately to obtain additional 
information. 

a. Results of adult literacy surveys conducted in Mississippi and Oregon. As 
previously noted, the Resource Group recommends that the Panel include information 
on the literacy of adults in these two states for the September 1991 Progress Report. 
While State - by-State data are not available to establish a baseline on the literacy skills 
of adults in .ill States, the data from Mississippi and Oregon are illustrative. These two 
States have conducted their own systematic literacy surveys, using the literacy 
assessment instrument developed by the Educational Testing Service for the Department 
of Labor. The data will be available for the 1991 Progress Report. 
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b. State estimates of persons needing literacy training. The Resource Group 
recommends that the Panel ask each State to estimate the number of persons who need 
or are eligible for literacy programs and the percentage of persons being served by such 
programs. The definition of literacy used for States should be broad, referring not only 
to basic literacy, but also to workforce literacy skills. 

c. General Educational Development (GED) awards. The Resource Group 
recommends reporting the number of GED certificates awarded in 1990 and annually 
thereafter. This information will be available for the September 1991 Progress Report. 
The group recommends that this figure be placed in context by also reporting the 
percentage of persons ages 18 to 34 who do not have high school diplomas or GED 
certificates. Another possible indicator is for State* to report the proportion of people 
taking the GED who failed to pass it. This information is also available in 1991. Even 
if high school graduation rates rise significantly in the coming decade, an increased 
number of GEDs awarded over the next 10 years will be a positive indicator of 
increased literacy. 

d. Reports of graduation rates from States that have student unit record systems at 
the college level For the September 1991 Progress Report, the Resource Group 
recommends that the Panel request States with student-based unit record systems and 
other viable record systems, such as special retention studies, institutional reporting 
systems, or coordinated institutional-based reporting studies to report on the overall 
retention and graduation rates at their public colleges and universities. Rates should be 
reported separately for 2-ycar and 4-year colleges. Where available, the information 
should be presented by racial/ethnic group. 

About one-third of the States have student unit record systems in place or in the 
final stages of implementation. Another dozen States conduct periodic studies of 
retention and degree completion by requiring institutions to submit aggre£ mc 
performance data. In both cases, information is only available on public ii.s'i.utions. 
About 15 states have no systems for reporting retention and graduation rates. While 
the 1991 Progress Report would not have information on all States, it is important to 
give greater emphasis to the information that is available and to encourage States 
without these systems to consider improving their retention and graduation data 
systems. 

e. State surveys on assessment of college seniors' critical thinking skills. Direct 
information on college seniors' performance is not available at this time. The most 
feasible way of directly determining these skills would be a national assessment 
targeted to a national sample of graduating seniors. Developing such an assessment 
would take much of the decade. For the present, the Resource Group recommends that 
the Panel ask each State to indicate whether it has systems to measure college students' 
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and graduates' ability to think critically, communicate effectively, and solve problems. 
States should also be asked to describe the systems briefly. 



HI. Data Being Developed for Reporting During the Decade 

The Resource Group on Goal 5 has several suggestions for the development of measures and 
indicators by the end of the decade. In the interim, assessments now in the implementation 
stage, chiefly the National Adult Literacy Survey (NALS), will become available in the next 
few years. The Panel should use these assessments until the measures envisioned in the 
future data system design (section TV) become available. 



A. Data for the Nation as i Whole 

i. National Adult Literacy Survey (NALS) 

The National Adult Literacy Survey is an assessment of knowledge and skills of adults. 
It '.vas developed by NCES under the jurisdiction of the Department of Education. The 
assessment is being field tested in 1991, will be administered in 1992, and can provide 
national information in 1993. States have an option to participate in the survey at a 
cost of approximately $350,000 spread over a 3-year period. While the survey is not a 
perfect measure of the adult literacy goal, it is a useful measure of progress and 
represents a substantial national investment. 

NALS is administered to a sample of households in order to estimate literacy. It 
does not yield scores for individuals. NALS will not and should not be used to 
measure the success of individuals in literacy programs, to evaluate individual literacy 
programs, to assess the instructional needs of individuals, or to design instructional 
programs. 

TJ .» law creating NALS calls for quadrennial surveys, in which case the next 
administration of the survey would occur in 1996, with data available in 1997. The 
Resource Group recommends that the survey be conducted as scheduled in 1992, that 
modifications suggested in section IV of this report be considered for adoption, and that 
the National Adult Literacy Survey be administered again in 1995 and 1998. 
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2. Definition of Worker Competency skills by the Department of Labor 

The Resource Group recommends that as information becomes available from the 
Department of La^r Secretary's Commission on Achieving Necessary Skills (SCANS) 
initiative, it should reviewed by *tc TVnel for possible inclusion in later reports. The 
DOL is developing a mcaso \a of the Nairn's adult training and education needs. 
DOL's SCANS program, orr ;ntiy unJe way, is intended to help define the necessary 
functional and higher Wtft uv's needed by entr '-level workers across a spectrum of 
jobs from manufacturing :« the service «t*7r. i lis effort will include defining 
competencies needed t » wee -d in high-pcriorra ince jobs. 



B. Data Being Developed for Sates 

1. Literacy Results for States Par:icipa.i:tt n the 1992 NALS 

As mentioned previously, States have the option to participate in the National Adult 
Literacy Survey. The scores from the States participating in the 1992 NALS should be 
included in the 1993 Panel report. Florida, Illinois, Iowa, and Texas are reportedly 
planning to conduct State-level adult literacy surveys in 1992. Other States are 
considering participating in the full-scale 1992 NALS. Efforts could be made to 
reduce the cost for States to participate in NALS by federal cost-sharing arrangements 
or concerted State efforts to secure private contributions to help offset costs. 

As other States participate in the 1995 and 1998 surveys, their results should also 
be reported. 

2. Other State Literacy Assessments 

All States should be conducting some sort of literacy assessments. If States are not 
participating in NALS, other alternatives need to be considered that would provide 
States with important information, including how a State's literacy levels and progress 
compare to that of the Nation. For example, States might develop shorter versions of 
the National Adult Literacy Survey, perhaps in conjunction with universities or other 
groups. States and the Department of Education could begin by looking at factors that 
make NALS more costly, and might consider options to reduce costs. The States and 
the Department of Education would need to determine whether the lower-cost version 
would be feasible and provide valid and reliable information on adult literacy. 

Another option is for States to adopt some procedure, such as the Comprehensive 
Adult Student Assessment System (CASAS), now being used by several States. 
CASAS has been used in States primarily for assessments of individuals in specific 
programs, i.e., welfare recipients. This option costs less than NALS because CASAS is 
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not administered to households. Hie Resource Group suggests that the Department of 
Education sponsor research to determine how CAS AS, similar literacy assessments, and 
the Graduate Equivalent Diploma (GED) can be equated with measures on the National 
Adult Literacy Survey. 

3, Use Graduation and Completion Rate Information Reported by Public and Private 
Postsecondary Institutions for the Student Right to Know Act 

The Federal Student Right to Know Act of 1990 requires all postsecondary institutions, 
public and private, including those that offer non-baccalaureate vocational programs, to 
report the completion or graduation rates of all certificate- or degree-seeking full-time 
students. Institutions are to begin recording and collecting this information by July 1, 
1991, and to complete reports by July 1, 1993. The law specifies that students may be 
counted as completing a program or graduating if they complete it "within 150 percent 
of the normal time for completion of or graduation from the program." They are also 
considered to have completed a program if they transfer to a progr m at another 
institution. 

The Resource Group recommends that the National Education Goals Panel plan on 
including information from the Student Right to Know Act when the system is fully 
operational. Our experience with data systems leads us to suspect that by 1994 or 1995 
the Student Right to Know Act may yield important and credible information for the 
private as well as the public sectors. 



IV. Future Data System Design 

The Resource Group recognizes the limitations of currently planned and available information 
on adult literacy and lifelong learning. Specifically, we recognize a substantial need to 
develop or modify assessment strategies in six areas: Enhancing the national adult literacy 
assessment in order to use it as an international indicator of workforce skills; developing 
benchmarks or targeted scores for the NALS; surveying businesses to determine the degree of 
their involvement in school-to-work transition and workforce training programs; 
commissioning periodic national polls on adults' perceived need for education and training; 
implementing student-based unit record systems at the postsecondary level in every State; and 
developing a national assessment for college seniors. 
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A. Recommended Future Measures 

1. Strengthen the National Adult Literacy Survey and Use It to Obtain International 
Comparisons of Workforce skills 

The Resource Group recommends that the Department of Education review NALS and 
develop a process for making any needed modifications and improvements. Any 
modifications should be ma le prior to the administration recommended for 1995. The 
Department of Education should consider issues such as the extent to which the 
National Adult literacy Survey (a) can provide useful literacy information about 
persons for whom English is their second language; (b) can be related to literacy skills 
needed ft r employment in high-performance workplaces; (c) can include additional 
mathematical measures not related directly to reading skills; and (d) can provide useful 
information about adults functioning at very low literacy levels. 

Americans should know how the functional literacy skills of their workforce 
compare with those of other developed nations. The adult literacy goal calls for "skills 
necessary to compete in a global economy." These skills may include the 
understanding and use of technology, problem solving, and the ability to learn. 
Knowing more about the skills of the American workforce compared with international 
measures will tell the Nation much about the competitive standing of the United States 
in the world economy. An enhanced National Adult Literacy Survey may provide the 
means for reaching agreements with other nations to measure cooperatively workforce 
skills. 

2. Developing Benchmarks or Targets for the National Adult Literacy Survey 

The Resource Group recommends that the Panel initiate a process to develop targeted 
scores for the NALS. The Panel should ask the Departments of Education and Labor to 
start the substantial research effort needed for this development. While the information 
to create target scores is not available now, we believe that it is possible to bring 
informed judgment and research to bear to establish target scores for performance on 
the National Adult Literacy Survey. The target scores should attempt to reflect the skill 
level i needed for American adults to function in the workforce and to take part in a 
representative government. Indeed, we are not recommending a single "cut point" on 
the assessment. Although developing several targeted scores for performance on the 
NALS will require a major effort, it mighfbe possible to complete this process by 
1995. 
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3. Surveying Businesses to Determine the Degree of Their Involvement in 
School -to- Work Transition and Workforce Training Programs 

As discussed above, the existing indicators of worker opportunity for education and 
training programs, namely the Department of Labor's periodic survey of workers' skills 
preparation and the Department of Education's National Household Education Survey, 
in reality measure the extent to which adults are taking advantage of opportunities. To 
assess the opportunities that are actually available, surveys of the business community 
need to be conducted. The Resource Group recommends that the Panel ask the 
Department of Labor to develop a systematic process for surveying a broad cross 
section of businesses across the Nation. 

The surveys would ask employers about the extent to which their business 
connects work experiences with the schools and offers transition opportunities, such as 
apprenticeships. They might also ask about employer satisfaction with new entrants 
into the labor market. The surveys should also be used to determine efforts by 
businesses to aid their workers in acquiring basic to highly technical knowledge and 
skills. Major investments are being made by businesses in training the workforce, but 
the Nation has little reliable information about these efforts. 

4. Information from Periodic National Polls on Public Need for Education and 
Training 

The Resource Group recommends that periodic national polls be commissioned to 
gauge the American public's perception of its needs for, and availability of, education 
and training opportunities, and the nature of these opportunities. The polls should be 
comprehensive enough to provide breakout information on major industrial sectors. 

5. Surveys of Adult Educational Programs in Postsecondary Institutions, Libraries, and 
Other Educational Institutions 

The Resource Group recommends that national and State representative sample surveys 
of adult education programs be commissioned two or three times during the decade. 
The surveys would be designed to report on the availability and effectiveness of such 
programs. 

6. Encouraging All States to Adopt Student-based Unit Record Systems or Comparable 
Systems to Track Student Enrollment, Retention, and Degree Completion in Public 
Postsecondary Institutions Statewide 

As discussed earlier, several States have developed or are implementing student-based 
unit record systems or similar processes for tracking students. The data provided by 
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such systems arc preferable to the indirect measures of graduation rates referred to 
above. The indirect measures are based on calculations from IPEDS. Student unit 
record systems provide more detailed information about student completion and 
graduation rates within a State. Individual enrollment patterns are tracked, in some 
instances using Social Security numbers. Students' college participation is followed 
regardless of whether they leave school for a time or transfer from one institution to 
another. The limitation of these tracking systems is that they are confined to the public 
sector and do not report graduation rates at private postsecondary institutions. The 
Panel should encourase all States to develop unit record systems. These systems 
should use common definitions and standardized reporting procedures that will yield 
generally comparable Uata from State to State. 

7. Assessing the Knowledge and Skills of Graduating Seniors 

As noted in the section on State data available for the 1991 Progress Report, neither 
national nor State information is currently available on the ability of college graduates 
to "think critically, communicate effectively, and solve problems." At present there arc 
no plans to develop a system for assessing the skills of a national sample of college 
graduates. It is possible that within the decade, individual States may develop 
measurement instruments capable of assessing these skills at the graduating senior level. 
For example, New Jersey has developed a test of General Intellectual Skills (GIT) that 
is administered to end-of-year sophomores. The possibility of wide-scale 
administration of a comparable assessment in several States appears remote. 

If the National Education Goals Panel wishes to attempt to assess the ability of 
college graduates to think critically, to communicate effectively, and to solve problems, 
a new kind of assessment will have to be created. That assessment might be a type of 
National Assessment of Educational Progress (NAEP) at the college level, given to a 
national sample of students at different kinds of institutions across the Nation. To have 
credibility, such an assessment would have to take into account the differences in the 
postsecondary institutions in America and the fact that the pluralistic system in place 
today has extended postsecondary educational opportunities to the broadest cross 
section ever of America's citizens. Developing a NAEP-likc collegiate assessment 
would be controversial for many reasons. It would require perhaps 5 years or more to 
develop and an investment of several scores of millions of dollars to make operational. 



B. Rejected Future Measures 

Use of the National Adult Literacy Survey (NALS) as a measure of college graduates' 
performance. The Resource Group recommends against using the NALS as a measure of 
college-level performance even though it is administered to the college graduates included in 
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the survey. There may be many reasons not to use the NALS to measure progress on the 
objective of college achievement. The charge of Objective 5 is to increase the advanced 
ability of college graduates to think critically. The most obvious reason for the Resource 
Group's negative recommendation is that using a literacy test that includes material at the 
elementary school level to measure the ability of college graduates' to "think critically, 
communicate effectively, and solve problems" is not appropriate. 



C. Summary 

The Resource Group for Goal 5 has outlined a systemwide agenda for indicators measuring 
adult literacy and lifelong learning for the September 1991 Progress Report and throughout 
the decade. This agenda includes collecting information on the range of literacy skills 
possessed by and expected of American adults, from basic literacy skills though skills need to 
function in a competitive workplace to higher-order literacy skills expected of college 
graduates. 

The Resource Group has also proposed CjX the National Education Goals Panel collect 
information on adults' participation in the education and training programs available in the 
Nation, including postsecondary programs, public employment and training programs, 
workplace programs, and others. 

The Resource Group also recommends that the Panel gather information on the extent 
to which opportunities arc available for adult learning programs across the States and Nation. 

Many gaps exist in the present data system; the Resource Group recommends that the 
Panel initiate a process to fill them. Overall, data arc more incomplete for States than for the 
Nation as a whole. Major recommendations for data development over the decide include 
strengthening the National Adult Literacy Survey while exploring ways to make it part of a 
cross-national effort to assess workers' knowledge and skills, and developing benchmarks for 
the NALS. We also note that a NAEP-likc examination of graduating college seniors would 
be necessary if the Panel is serious about measuring the advanced thinking, communication, 
and problem-solving skills of the Nation's college graduates. 
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Endnote 

1. A concern with the 1985 assessment is that individuals with limited English proficiency or 
levels of reading so low they could not complete the survey were excused from taking the 
assessment. Thus, some experts argue that the present survey slightly overestimates (by about 
2 percent) the literacy level of young adults. 
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Chapter 6 

Safe, Disciplined, and Drug-Free Schools 

An Interim Report from the Resource Group on Safe, Disciplined, and Drug-Free Schools 



GOAL 6: By the year 2000, every school in America will be free of drugs and violence 
and will offer a disciplined environment conducive to learning. 

Objectives: 

• Every school will implement a firm and fair policy on use, possession, and distribution 
of drugs and alcohol. 

• Paicnts, businesses, and community organizations will work together to ensure that 
schools are a safe haven for all children. 

• Every school district will develop a comprehensive K-12 drug and alcohol prevention 
education program. Drug and alcohol curriculum should be taught as an integral part 
of health education. In addition, community-based teams should be organized to 
provide students and teachers with needed support. 
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In early 1991, a Resource Group on Safe, Disciplined and Drug-Free Schools was convened by 
the Panel to recommend indicators and strategies for measuring progress toward achieving this 
goal. Members of the group are as follows: 



John W. Porter 



Detroit Public Schools, Detroit, Michigan (convener) 



C. Leonard Anderson 



Portland Public Schools, Portland, Oregon 



Constance E. Dayton 



Philadelphia Public Schools, Philadelphia, Pennsylvania 



Delbert G. Elliott 



University of Colorado, Boulder, Colorado 



Joseph A. Fernandez 



New York City Public Schools, New York, New York 



Michael Guerra 



National Catholic Educational Association, Washington, D.C. 



J. David Hawkins 



University of Washington, Seattle, Washington 



Lloyd D. Johnston 



University of Michigan, Ann Arbor, Michigan 



The Panel and Resource Group welcome your reactions to the Interim Report of the Resource 
Group on Safe, Disciplined, and Drug-Free Schools. 
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National Education Goals 
Goal Six 

Safe, Disciplined, and Drug-Free Schools 



L Introduction 

The goal of safe, disciplined, and drug-free schools carries great importance to American 
young people and the American public at large. The country is now more than 20 years into 
a costly and tragic epidemic of illicit drug use — one that has probably not been replicated in 
any other country in the world — and both students and the public have rated it among their 
top concerns for many years. Safety is another concern of obvious and considerable size, as 
is the need for sufficient discipline in the schools so that effective teaching and learning can 
take place. 

This goal is important in its own right, of course, but it also has important inter- 
relationships with the other five educational goals adopted by the Nation. For students to be 
motivated, clear-headed, and attentive to the educational tasks before them, their minds and 
bodies must be free of the effects of alcohol and other mind-altering substances. Students 
must also be free of the fear of being the victims of assault and other criminal acts. Further, 
an orderly environment, as opposed to a disruptive and chaotic one, is needed to enable 
young people to fulfill their academic potentials. Hie closer we come to attaining these three 
elements of Goal 6, the more likely it is that we will see improved academic performance, 
more students staying in school, and more of them attending regularly. 

Conversely, attaining some of the other goals may help in attaining Goal 6, since poor 
performance in school — usually beginning at an early age — is a significant predictor of 
young people's involvement with drugs, delinquency, and disruptive behavior. To the extent 
that early readiness for school can be achieved, to the extent that young people do better in 
their coursework and thus experience success, and to the extent that they are less motivated to 
leave school, we are lixely to observe lowered drug use, less violence and crime, and less 
disruptive behaviors among them. 
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n. Definitions and Assumptions 



A. The Three Elements of Goal 6 

There are three distinguishable elements within Goal 6: drug-free schools, violence-fire:! 
schools, and an orderly environment conducive to learning. The Resource Group has chosen 
to address each element separately and to interpret these elements as follows: 



1. Drug-Free Students and Schools 

While a literal reading of the goal suggests that only the schools are to be drug free 
(which already appears to be largely the case according to research results), we believe 
that it is more important that the students are drug free, irrespective of where and when 
they use drugs. If they arc coming to school high, hung over, or with other lingering 
effects on their alertness, cognitive functioning, or motivation, then they are functioning 
at a below-normal level in school. Thus, the emphasis here is on the students, not just 
on their in-school behavior. 

Unfortunately, American students now have before them a wide array of chugs 
both licit and illicit, from which to choose. All these drugs adversely affect students' 
cognitive functioning, social and emotional maturation, physical health, and likelihood 
of becoming involved with increasingly dangerous drags. Therefore, the domain of 
drug-using behaviors should be defined broadly to include not only the clearly illegal 
drugs, but also psychotherapeutic drugs used outside of medical supervision, as well as 
inhalants, steroids, alcohol, and cigarettes. While not taken for their psychoactive 
effects, anabolic steroids can have emotional and behavioral effects as well as serious 
health consequences. Cigarette smoking, while unlikely to impair academic 
performance directly, docs serve as a "gateway" behavior to the use of other drugs that 
do have such effects. Further, the schools have an important educational role to play in 
preventing the onset of smoking — most of which begins in the teen and preteen years- 
-since a continuation of current smoking patterns among American young people will 
lead to the early deaths of literally millions of the children in school today — as many 
as 5 million by some estimates. 

2. Schools Free of Violence and Crim ?. 

* 

While the goal mentions only violence in its brief wording, the Resource Group 
interpreted the goal to mean schools free of crime more generally, including the threat 
of violence as wel# as theft and vandalism. Because carrying weapons is closely 
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associated with committing the most serious forms of crime, that behavior has been 
included in the definition of this element of Goal 6, as well. 

In contrast to the case for drugs, here it is proposed that the focus should be 
confined to the domain of the school, since being victimized is an experience, not a 
behavior, and the school has little influence on this experience beyond its own domain. 
That domain, incidentally, is interpreted as including being in or around the school, as 
well as being in transit to or from school, particularly on a school bus. Acts of 
violence and crime are defined as including acts against the person (e.g., armed and 
unarmed assault, threats of armed and unarmed assault, sexual assault) and acts against 
property (e.g., major and minor theft, vandalism). 

Unlike most of the other five goals, this element of Goal 6 could be interpreted is 
applying to school faculty and staff as well as to students. No proposal was offered tj 
measure drug use among school personnel, although a reasonable case for it might be 
nadc; but it was felt that the victimization experiences of school personnel should be 
included in the measurement, not only because of their intrinsic importance for the 
teachers, but also because such experiences, or fear of their occurrence, can impair 
teaching performance and thereby have a considerable derivative impact on student 
learning. 

3. Disciplined Environments Conducive to Learning 

This element in Goal 6 has proven the most difficult to define precisely. There is 
. agreement about what it is not: It is not synonymous with order and quiet in the 
classroom. Youngsters in an active, at times boisterous, classroom may be learning 
very effectively, and discipline may be expressed in less readily observable ways. For 
this reason, it was felt that students and teachers themselves may be in the best position 
to judge the extent that their learning and teaching environment is conducive or 
inimical to learning. More concrete measures of behaviors and events may also be 
used as indicators of a disciplined environment; for example, evidence of disrespect, 
verbal abuse, and physical abuse in teacher-student and student-student relations would 
be negative indicators, and an active and constructive interface between the school and 
parents might be considered a positive indicator. Some Resource Group members felt 
that high rates of school attendance, by both students and teachers, as well as student 
punctuality for class, should also be included as positive indicators. 

B. The Three Instrumental Objectives for Goal 6 

Three objectives are outlined by the Goals Panel as instrumental to attaining Goal 6. The 
Resource Group for Goal 6 focused considerable discussion on the appropriateness, relative 
importance, and completeness of these objectives. While acknowledging that they deal with 
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important dements for obtaining the goal, the Resource Group noted some reservations which 
are outlined below. 

i. School Policy on Drugs and Alcohol 

The first objective states, "Every school will implement a firm and fair policy on use, 
possession, and distribution of drugs and alcohol." The 1990 amendments to the Drug- 
Free Schools and Communities Act require that by October 1990 drug policies and 
education about the use and possession of drugs be in place if school districts are to be 
eligible for any Federal education assistance. These conditions make it likely that this 
objective is already obtained, setting aside the qualifiers in Goal 6 of "firm" and "fair." 
(Note that defining "firm" and "fair" for measurement purposes would be highly 
subjective and therefore difficult.) In any case, the U S. Department of Education will 
be gathering data on 'xmipliancc, and no new measurement appears required to measure 
the existence of policies. 

But writing a *x>licy is only part of a more complex process involving the way the 
policy was developed (which can affect whether parents and students see it as 
legitimate), the extent to which it has been communicated clearly and fully to students 
and parents, the extent to which students see it as being enforced and taken seriously 
by teachers and other school authorities, the extent to which teachers and other staff 
have been trained to recognize symptoms of use and are able to intervene effectively 
within the framework of the policy, the extent to which assessment services and referral 
services (which should be part of the policy) are available and used by the school, and 
so on. Thus, measuring only the existence of the policy presents a rather limited 
instrumental objective compared with the full complement of elements necessary for the 
policy to make a difference in students' use of drugs. 

The Department of Education's 1991 First Response School Survey (FRSS) of a 
national sample of districts, principals, and teachers provides some measures of the 
extent to which students, parents, and various sectors of the community were involved 
in developing of alcohol and other drug policies (in the district and principal 
questionnaires), the extent to which teachers and administrators have received relevant 
training (in the principal and teacher questionnaires), and the extent to which various 
student assistance services arc available (in the principal questionnaire). This survey 
also contains other measures of relevance (including whether the district has a written 
policy). All of these elements might be considered in measuring progress toward this 
instrumental objective, should the Goals Panel be inclined to broaden its definition of 
the objective. One or more repetitions of the survey could be considered if it proves 
particularly helpful. The extent to which students are aware of the policies and believe 
that they arc enforced could be measured longer term in the Monitoring the Future 
study of annual national surveys of 8th, 10th, and 12th grade students and in CDCs 
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biennial state level surveys of students in grades 9 through 12. (Both of these survey 
series are described further below.) 

2, Parent and Community Involvement 

The second objective reads, "Parents, businesses, and community organizations will 
work together to ensure that the schools are a safe haven for all children." This 
instrumental objective addresses only the violence-free component of Goal 6, although 
the role of the community in drug abuse prevention is acknowledged later in the third 
objective. Clearly, the potential contributions of parents and the community are very 
great, not only for achieving the goal of disciplined, drug-free, and violence-free 
schools, but also for achieving nearly all of the six educational goals. At the end of 
this report we suggest that parent and community involvement might be raised to the 
level of super-objectives, relevant to achieving all goals. 

The 1991 Fast Response School Survey mentioned above also contains measures 
of the extent to which parents and various other sectors of the community assist schools 
specifically in support of their alcohol, tobacco, and other drug prevention programs. 
This information is reported by principals and also by whoever completes the district- 
level questionnaire. Presumably such questions could be included in future FRSS 
special surveys. While there are no comparable questions asking to what extent these 
other groups help to bring about a safe environment in the school, such questions could 
be added to future such surveys to mark progress on this instrumental objective. 

3, A K-12 Prevention Education Program 

This objective reads, "Every school district will develop a comprehensive K-12 drug 
and alcohol prevention education program. Drug and alcohol curriculum should be 
taught as an integral part of health education. In addition, community-based teams 
should be organized to provide students and teachers with needed support." 

The need for a comprehensive K-12 prevention program is acknowledged. In 
fact, the National Commission for Drug-Free Schools (1991) report specifically 
recommended the program. However, emphasis needs to be given to two words: 
comprehensive and effective. Comprehensive should be interpreted to mean a program 
comprising not only elements in the formal curriculum, but also a well-thought-out 
school policy (already discussed above), a student assistance program, formal in- 
service training for teachers (and eventually such training required for certification), 
assessment program, organizational component to deal with drugs, and so on. Effective 
means that the program, or at the least the elements in the program, are of 
demonstrated effect . The same National Commission report stressed both of these 
points. Many schools have auricular programs of unproven effectiveness or proven 
ineffectiveness; the mere existence of programs is not necessarily a good thing—it may 
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even reflect a waste ok resources. Many questions on effectiveness remain unanswered 
because funding for evaluation has been meager until the past couple of years, but the 
existing knowledge should be used in whatever program selection or development takes 
place. 

Teaching a drug and alcohol curriculum as part of health education mag be a good 
approach, but the Resource Group feels that it is far too early to name this curriculum 
as the only good approach, or even the best approach, as Objective 3 seems to do. An 
infused curriculum or specialized curriculum may be as good or better. There is as yet 
inadequate evidence upon which to begin homogenizing the country's approach to 
alcohol, tobacco, and other drug prevention. 

As the National Commission for Drug-Free Schools observed, while problems of 
drug use among our students may be noticed or measured most often when young 
people are in school, many of the influences that cause drug use are well beyond the 
boundaries of the school. The family, church, local community, public role models, 
and the media all play a role in the etiology of drug abuse, and, therefore, all have a 
potential role to play in preventing it. The same may be said for the other behaviors of 
special interest here — violence and other crime, and disruptive behavior in school. 

Thus, while the schools can and should do a considerable amount to address these 
problems among our youth, the whole burden cannot be left to them alone. Other 
sectors, in particular parents, local service and enforcement agencies, and the 
community at large, arc needed as active and invested participants. The schools can 
play an important role, however, in helping to mobilize and inform these related 
sectors. 

The National Commission recommended a somewhat different model than the one 
proposed as the third instrumental objective by the Goals Panel. The commission 
recommended that each school establish a permanent task force to deal with alcohol 
and other drugs; that it be made up of parents, social service agency representatives, the 
police, etc., in addition to students, teachers, and school administrators. Further, the 
commission urged school districts to help launch community-wide leadership groups, 
which would work independently of the schools but coordinate their efforts with them. 
Indeed, a number of communities around the country have already launched such 
community coalitions, including Miami; Washington, DC; Rochester, NY; and Kansas 
City. 

The exact form of such efforts may not be as important as the fact that various 
elements in the community are mobilized to work in concert with the schools in helping 
to prevent the use of alcohol and other drugs, and that coordination between the school 
and community service agencies has been achieved. Thus, a broader measure than that 
suggested by the objective as currently written would appear to be justified. 

The 1991 Fast Response School Survey contains a list of nine possible 
components of the schools' drug prevention program, which could provide useful 
information on the degree to which the various elements are present in schools. This 
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information is provided on the district-level questionnaire, which may mean somewhat 
lower validity in the answers than could be obtained from principals, particularly if the 
programs vary by school within the district. For longer-term purposes, these questions 
should probably be asked of principals. 

The extent to which drug prevention is taught in all grades, as Objective 3 
encourages, can be measured in a question in the FRSS district-level questionnaire. 
The questionnaire also contains a question on where in the curriculum drug education 
resides at the elementary, junior high, and high school levels. "Within the Health 
Curriculum" is one of the answer categories. 



In sum, it was felt that a better alignment between the goals and the objectives is 
possible, and this possibility would influence the future of what is measured. Clearly, 
priority should be given to measuring progress on the goal itself, but some useful indicators 
could be gathered with regard to the objectives, as well. Nearly all of the objectives, 
however, suffer from the fact that the mere presence of the element (policy, curriculum, 
community group) docs not mean that the element is either functioning or effective. 
Therefore, merely measuring their presence may lead to a false senje of progress, when in 
fact the situation may be stationary or even deteriorating. On the other hand, more 
penetrating measures are more difficult to develop and apply. 

Finally, some members of the Resource Group felt that of more importance than the 
instrumental goals stated may be some that are not stated. In particular, objectives aimed at 
changing some of the risk factors (or intervening variables) may need to be influenced to 
bring about changes in drug use, delinquency, or disruptive behavior in school. Such risk 
factors as not bonding to the school, having high absentee rates, going out on school nights, 
and not understanding the hazards of drug use are seen as particularly relevant. 



C. The Universe of Students of Greatest Concern 
1. Age Range 

For Goal 6, the recommendation is that, for the most part, the measurement effort 
should be focused on the secondary school levels — that is, middle schools, junior highs, 
and high schools. This recommendation is based largely on the fact that the behaviors 
of alcohol and other drug use, and the experiences of being the victim of violence or 
other crimes, tend to be concentrated in these age groups. There is also a practical 
reason, however; gathering data on these subjects directly from students grows more 
difficult as one moves down to the primary school ages. Quite appropriately, school 
personnel and parents are more protective of younger students, and the risk of the 
questions suggesting behaviors that might not have been considered previously grows. 
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Thus, it is suggested that data gathered on the behaviors and experiences of younger 
children be confined mostly to those reported by adults in the schools or to 
retrospective reports of behaviors at earlier ages gathered from students at the middle 
school level. 

2. Public vs Private 

An important segment (and potentially shifting proportion) of the student population 
attends parochial and other privately run schools. The recommendation is that these 
schools be included in the domain of measurement at both the national level and, when 
possible, at the State level. 



HI. The 1991 Progress Report 



A. Drug-Free Students and Schools 
L National Indicators 

a. Recommended indicators and data sources. Because of the broad array of factors 
and outcomes that might be measured with regard to drug use and because measures for 
three different elements under Goal 6 are needed, it was concluded that a few selected 
measures should be emphasized, with particular emphasis on outcome measures. 
Primary would be direct measures of the students' use of drugs. Second would be 
measures of the peer groups' prevailing norms regarding drug use, which are important 
because they provide the bulwark for resisting use in the future. Eventually, measures 
of the frequency of being under the influence of alcohol or other drugs while at school 
and of the extent of drug sale or distribution at school would be desirable. However, 
measures of these latter variables must still be developed and implemented. In the 
interim a measure of use while in school is available and could be used. 

(1) Student use of drugs. The Monitoring the Future (MtF) surveys of 
American high school seniors contained the most comprehensive measurement of 
drug use among American secondary school students to date. Tias series of 
annual national surveys, which is being expanded to include 8th and 10th grade 
students in the spring of 1991, has data available on national samples of 12th 
graders over the past 16 years (N=16,000/yr.) and, thus, provides the opportunity 
to put the changes during the present decade into longer-term historical 
perspective. Because its measures of drug use also have been used to develop 
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national measures in other countries, particularly in Western Europe, the MtF also 
offers the possibility of putting the American results into international perspective. 

The 8th and 10th grade data will not be ready in time for the September 
1991 Progress Report, but they will be by 1992. Meanwhile, data from a national 
survey of 8th and 10th grade students in 1987, using highly comparable measures 
to MtF from the National Adolescent School Health Survey (NASHS), could be 
used to provide some benchmark measures. 

The MtF survey contains measurements for the use of any illicit drug othfii 
than marijuana, as well as for a host of specific drugs including marijuana, 
cocaine, crack, hallucinogens, LSD, heroin, "ice," inhalants, and the nonmedically 
supervised use of stimulants, sedatives, tranquilizers* and opiates. Alcohol and 
tobacco use are included, as well as anabolic steroid use. The study also has 
questions on the grade in which the student first used these drugs. When applied 
to 8th grade students, this grade of first-use measure can provide a time-lagged 
measure of drug initiation rates at lower grades. For example, the 1992 data from 
8th graders could be used to calculate the lifetime prevalence for 6th graders in 
1990. This type of approach is recommended for estimating drug use levels at 
lower grade levels. Because of the broad array of substances abused by American 
school children and the likelihood that still others will emerge, such 
comprehensive coverage is important, even though some simplifying indexes will 
probably be needed for the Progress Report itself. 

(2) Peer norms. Among the environmental and psychological measures in the 
Monitoring the Future study directly related to drug use, are the perceived atitudes 
of friends concerning the use of the various licit and illicit drugs. The answers 
are given in terms of the degree of peer disapproval that could be expected in 
response to various drug-using behaviors. Again, these excuses have been 
monitored for 16 years, during which considerable changes have taken place. 
These norms set the first line of defense against any renewed epidemic use of 
these drugs. 

(3) Drug use in school. The Resource Group felt this behavioral measure should 
be replaced in the longer term with a measure of being under the influence of 
alcohol or other drugs at school, which was judged to be the most relevant for 
school performance and more important than tthfilfi the drugs are actually taken. 
In the interim, however, a set of measures on the prevalence of use during the past 
year at school exists for alcohol and other drugs in the MtF data set and could be 
reported through 1990. Generally speaking, these rates tend to be quite low. 

b. Indicators and data sources considered but not recommended. A number of 
predictors of use or risk factors were considered for inclusion, but were judged less 
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important than those listed above. However, many are important social and 
psychological factors judged by the Resource Group to be important in attaining the 
outcomes just discussed. Among the psychological risk factors* seriously considered 
were students' beliefs about the dangers of using various drugs (which have proven to 
be important determinants of use), their judged ability to resist peer pressure to use, 
their stated willingness to use, and their norms against deviant behavior generally. 
Among the environmental risk factors judged to be of particular importance were the 
perceived availability of drugs, the extent of active parental monitoring of behavior, and 
the availability of counseling, referral, and student assistance programs. A number of 
these variables are currently measured in the MtF study and could be added as 
indicators here, should that be judged desirable. 

As to measures not used for the three outcome variables that were chosen for the 
Progress Report, the use of systems that ask students or teachers to report their 
judgments of the extent of drug use, for instance, was not felt to be as reliable or 
accurate as direct self-report measures. Thus, these measures are not recommended. 

The Youth Risk Behavior Surveillance System (YRBSS) does provide self-report 
measures on a numb r of the drugs of interest here, using a subset of questions drawn 
largely from MtF. This annual national survey of students in grades 9 to 12 was begun 
in 1990 and provides directly comparable results between its National- and State-level 
samples (which occur biennially on odd-numbered years in a limited number of States 
so far). YRBSS, however, provides data on smaller national samples (N=12,000/yr.) 
and on a more delimited set of drugs than MtF, and thus was judged to provide 
measures on this goal that would not be as comprehensive or quite as stable as those 
from MtF (N=48,000/yr.). YRBSS is recommended as the best available vehicle for 
measuring progress at the State level, however. (Sec below.) 

Consideration was also given to the desirability and feasibility of monitoring the 
alcohol- and other drug-using behaviors of faculty and staff in the schools. 
Monitoring is seen as highly relevant both because such use may impair teaching 
performance and because teachers and other school staff serve as important role models 
for their students, and an observable contradiction between their behavior and their 
prevention messages will obviously undercut the latter. However, the feasibility of 
surveying teachers regarding their own behaviors was judged to be low, since 
cooperation problems could be expected and a whole new level of expensive data 
systems would need to be mounted. Therefore, ongoing Department of Education 
surveys might include more general questions to determine the extent to which 
respondents see teacher or staff behavior being impeded by the use of alcohol or other 
substances. Such measures could provide a more general and indirect measure of the 
problems. 



110 

11Z 



Resource Group Interim Reports 



2. State-Level Indicators 

a. Recommended data sources 

(1) Student use of drugs. Because there are 50 States, the measurement costs of 
adding new data collections on 50 representative State sampU s are very 
considerable, not only in terms of dollars, but in terms of student and staff time. 
For this reason the Resource Group has leaned heavily toward using existing data 
collection systems, particularly at the state level, and particularly toward ones that 
are comparable with the recommended national data collection system (or at least 
have that possibility). In the area of substance use, CDCs Youth Risk Behavior 
Surveillance System seems most promising, because CDC is encouraging and 
helping States to apply the survey instrument at the State level. This instrument 
already contains some drug use m asures drawn from Monitoring the Future, 
although the list is not as comprehensive as MtF's and has been slightly revised. 
YRBSS has questions on the use of marijuana, cocaine, crack, other illicit drugs 
taken as a whole, steroids, alcohol, and cigarettes. While there may not be room 
for much expansion of the coverage (for example, to include hallucinogens, 
inhalants, heroin, other opiates, "ice," and the various psychotherapeutic drugs), 
the key drugs included are probably adequate for assessment purposes at the State 
level. 

Not all States are using this survey, which is administered biennially on odd- 
numbered years, and only limited standardization of survey procedures has as yet 
been attained at the State level. However, the survey was conducted in 30 States 
during 1990, its first year, with half of the participating States selecting 
representative samples. CDC is encouraging the participation of all States and the 
improvement of survey procedures over time. The Resource Group would 
encourage the states to work toward more standardized and rigorous procedures to 
develop data that can be meaningfully compared across time and against the 
national data. 

Many States may want to report these drug use data for the 1991 Progress 
Report, although it should be added that a number of States have their own 
statewide surveys of student drug use — many using measures drawn from MtF. 
Those that have recently conducted their own surveys of student drug use may 
prefer to base their reporting on those surveys, although comparability with the 
MtF national results will vary by State. 

(2) Peer norms. YRBSS does not now have questions on this subject, nor does 
any other widely used State-level survey. Some States may have adopted these 
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questions from MtF in their own drug abuse surveys, but these will be the 
exception. 

(3) Use of drugs at school. The same situation exists for these measures as for 
peer norms. 

b. Data sources considered but not recommended. Some measures have had 
widespread use, but usually are not applied to representative State samples. For that 
reason and others, these measures have not been recommended here. However, if these 
measures have been applied to a representative sample in some States, those States may 
wish to use them to measure their progress on this element of Goal 6. 



B. Schools Free of Violence and Crime. 

1. National Indicators 

Fear for personal safety may become an overriding concern for staff and/or students, 
inhibiting teaching, learning, and even attendance. Thus, it is important to measure 
staff and student feelings of personal safety in school and in the school neighborhood 
that must be traveled to get to or from school. An adequate assessment of the degree 
to which schools arc free of crime and violence requires measuring the incidence of 
victimization actually experienced by staff and students at school and on the way to and 
from school. Weapons in schools, whether carried to reduce fears for personal safety 
or for other reasons, represent a threat to the actual safety of the other members of the 
school community. Therefore, measuring the extent of weapons possession in school is 
recommended. 

a. Recommended indicators and data sources. 
(1) Victimization in school. 

Students. MtF contains a battery of seven questions in which students report the 
frequency with which they have been the victims of crime and violence in school 
(or on the way to or from school) during the prior 12 months. These questions 
cover being the victims of major and minor theft, vandalism of one's property, 
injury and threat of injury with a weapon, and unarmed injury or threat of injury. 
Long-term trends in these variables can be reported up through 1990 for high 
school seniors and, starting in 1991 surveys, comparable measures will be 
available for 8th and 10th grade students. There is also a measure of vandalism 
of school property in MtF that should be reported along with personal 
victimization measures. 
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Teachers . Regarding the victimization of faculty in the school, in the spring of 
1991 the U.S. Department of Education's Fast Response Survey System (FRSS) 
will carry out a national survey of school systems which deals with the themes in 
Goal 6. The teachers questionnaire contains six questions of direct relevance here. 
These questions ask the lifetime prevalence of verbal abuse, threat of attack, and 
attack by students, and the frequency of these same experiences in the prior 12 
months (in the case of attack or threat of attack) and in the last 4 weeks (in the 
case of verbal abuse). FRSS is described as a one-time policy-oriented survey, 
but some of its results will be very useful for the 1991 Progress Report; the most 
relevant items could be included in future Schools and Staffing Surveys (SASS) in 
the teacher questionnaire, or might be included in a subsequent FRSS survey 
devoted to this topic again. 

(2) Feeling safe at school. 

Students. The School Crime Supplement (SCS) to the National Crime Survey 
conducted in 1989 contains some highly pertinent questions asked of students, 
including the frequency of fear of attack at school and on the way to school, and 
the frequency of staying at home because of fear of attack at school. Finally, SCS 
asks whether various areas in the school, or on the way to or from school, are 
avoided because of the fear of attack. Students aged 12 to 19 (N= 13,000) are 
asked these questions in a household survey. These data should be included in the 
Progress Report. 

Teachers . FRSS contains four questions that ask teachers how safe they feel in 
the school building during and after school hours, on the school grounds, and in 
the school neighborhood. These questions would be quite appropriate to include. 

(3) Carrying weapons to school. The 1989 SCS asked students the frequency 
with which they nad brought weapons to school over the prior 6 months and the 
types of weapons they brought. Assuming that the answers to these questions, 
which were asked in an interview situation, are valid, the answers should measure 
the information needed quite directly. 

b. Considered but not recommended. Administrative reports of school safety, 
victimization, and weapons possession from buildings and districts are influenced by 
many factors other than the actual prevalence of these problems. Therefore, it is 
recommended that to the extent possible, the data used in both the 1991 Progress 
Report and in the long-term data system not be based on administrative reports but be 
gathered directly from probability samples of teachers and students. 

Continuing survey series have been generally preferred over one-time surveys 
when their measurement coverage was roughly comparable, because the survey series 
will allow the Goals Panel to stay with the same indicators longer term. One example 
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of a noncontinuing survey is the School Crime Supplement to the National Crime 
Survey, which has been conducted only once so far, in 1989. like MtF, this survey 
asked about the frequency of robbery involving weapons, threat of force, or use of 
force while at school; the frequency of theft of possessions at school; and the frequency 
of physical attack at school and physical attack requiring medical attention. (The 
timeframe for these questions was the prior 6 months.) However, because it is not 
clear whether, or how frequently, the SCS might be repeated, the SCS was not 
recommended for use here. There is also some concern about the validity of such 
information when gathered from children in a household setting with parents nearby. 



2. State -Level Indicators 

The data systems just recommended as sources of national data do not generate data on 
State samples, nor do they have counterpart studies at the State level. Thus, no 
measures regarding violence and crime, feelings of safety, or carrying weapons can be 
included at the State level for the 1991 Progress Report. 



C. Disciplined Environments Conducive to Learning 

I. National Indicators 

To define what constitutes a "disciplined environment conducive to leanrng" is a 
challenging task. Some argue that it should incorporate factors such as the physical 
environment, e.g., whether the school buildings are clean, bright, well-maintained, and 
adequate in size. Others argue that is should include factors related to the 
organizational climate, e.g., whether a positive and constructive relationship exists 
between faculty and administrators, faculty and parents, faculty and students, and 
among the various members of the faculty. Still others argue that student and teacher 
absenteeism, student tardiness, and homework completion by students are indicators of 
an orderly environment. Nearly everyone would agree that the learning conditions 
inside the classroom during school hours, including excessive noise, fighting, and other 
disruptions, should be included. 

The Resource Group was not able to reach consensus on how broadly to define 
this element of Goal 6. Consensus was reached on specific issues, however. First, 
there is full agreement about the importance of measures based on perceptions as well 
as behaviors, since students and teachers themselves may be the best judges of whether 
conditions in the school environment are conducive to or disruptive of their learning 
and teaching, respectively. 
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With regard to behaviors, student misconduct would be measured and reported in 
terms of the frequency of definable events. Measures should include reports of class 
disruption, teacher harassment, and other items common to most student conduct codes. 

Some members of the group recommended inclusion of school attendance, class 
cutting, and tardiness as appropriate indicators for this goal, but others argued that 
conceptually these behaviors are not measures of a disciplined environment per se. 
These issues are raised again in Section IV of this report under a discussion of "super 
objectives.** 

Research by the National Association of Secondary School Principals School 
Climate Task Force suggests including measures of student productivity as well as 
measures of student/teacher satisfaction in descriptions of the school environment. The 
Resource Group did not reach consensus on this issue. Some place primary emphasis 
on "disciplined environment"; others underscore the importance of "conducive to 
learning" and would recommend the included student productivity measures such as 
completion of homework and in-class assignments, time on task, and other student- 
engagement measures. 

a. Recommended indicators and data sources 

(1) Student misconduct in school. To some extent there is an overlap between 
this element of Goal 6 and the previous one dealing with schools being free of 
violence and crime. Events such as verbal abuse of teachers by students, student 
threats of injury to teachers, and actual physical attacks on teachers are all 
relevant to defining an orderly environment conducive to learning. Not only is 
teacher safety affected by such misconduct, but the learning environment for other 
students is, as well. 

Additional elements of student misconduct can be measured, of course. 
Students can be asked directly about such behaviors as not cooperating with 
teachers, not following instructions in class, being disruptive in class, cheating on 
tests, copying other people's homework, verbal and physical altercations with other 
students in class, and so on. There can be little doubt that these features of the 
school environment are not conducive to learning, either by the "perpetrators" or 
by the other students in the class. At present, no single data source appears to 
have a good set of measures of these student behaviors. MtF docs contain two 
questions regarding the extent to which student norms support cheating and 
noncoopc ration with teachers: These are available only for high school seniors 
until after 1991. The National Educational Longitudinal Study for 1988 
(NELS:88) contains questions in its 1990 followup survey about whether students 
feel that it is "OK" to cheat on tests, copy other people's homework, cut classes, 
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cut school, or be late for school. The study also has two questions on 
misbehavior in class by other students and one on whether they tend to get away 
with misbehavior. These questions could be used in the 1991 Progress Report, 
although the study is not appropriate for long-term monitoring. NELS:88 is a 
panel study of a national sample from a single-class cohort who were in 10th 
grade at the time of the 1990 data collection. 

(2) Student and teacher perceptions of noise and disruption impeding 
learning. The most desirable method for measuring change across time in student 
and teacher perceptions of what ihey. define as noise and disruption is to ask them 
about the frequency of such occurrences. Such a measurement approach generally 
has not been adopted to date in student and teacher surveys that provide national 
data. NELS:88 has two questions in its student questionnaire that ask, on an 
agree-disagrec scale, whether "other students often disrupt class" and whether 
"disruptions by other students get in the way of my learning." 

The teacher questionnaire for the 1991 FRSS asks teachers to what extent 
student misbehavior and student disruptive behavior interfere with their teaching. 
There are also questions on how serious student tardiness and student 
absenteeism/class cutting are as problems in the school. Again, none of these 
questions request frequency of occurrence, which would be preferable for 
measuring change over time because the meaning of the scale is more likely to 
remain constant; but they still could provide interesting and useful data for the 
1991 Progress Report. 

(3) Other indicators. If the Goals Panel decides to include measures of 
tardiness, skipping school, skipping class, hours spent on homework, or homework 
completion in the indicators of this element of Goal 6, some national measures do 
exist. MtF has measures for high school seniors in the class of 1990 (and for 15 
senior classes prior) of the frequency of skipping whole days of school and 
sapping classes. MtF also measures hours per week spent on homework. 

b. Considered but not recommended. Again, information from administrative reports 
was considered and rejected. Simply collecting administrative data on the number of 
violations of school codes would provide an unreliable measure of student misconduct, 
because school codes vary across schools, systems, and time. 

2, State-Level Indicators 

None of the data sources recommended for use at the national level have counterparts 
at the State level now. The only data collection system with relevant State-level data 
is the Schools and Staffing Survey. The SASS teacher questionnaire contains a few 
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questions about the level of student misbehavior, the consistency of rule enforcement 
by teachers, and tardiness and class cutting by students interfering with teaching. 
Unfortunately, these questions are on an agree-disagrec scale, which makes 
comparisons across States or over time difficult, because teachers' standards can differ 
or change with time. The same type of problem exists for a set of questions on student 
tardiness and absenteeism, teacher absenteeism, students cutting class, and fights among 
students. These problems are all rated on a scale measuring how "serious a problem" 
each is. Nevertheless, because these questions are available on national and State levels 
(only public schools are included at the State level), the panel may wish to repoit the 
results on these three questions in the 1991 Progress Report. At least they give some 
idea of how the teachers see things. 



IV. Design for a Longer-Term Data System 

As the discussion in the last section indicates, there are clear needs for improving existing 
data systems if progress toward Goal 6 is to be measured accurately and completely at both 
the national and State levels. While substance use and student victimization in school are 
fairly adequately measured at the national level in ongoing data systems — in particular, 
Monitoring the Future — this measurement might be expanded somewhat. Substance use is 
not, however, being measured completely at the State level in terms of coverage of all states, 
coverage of all the indicators, or comparability and adequacy of measurement procedures 
across States. Victimization in school is not being measured at all at the state level, as best 
we can determine. 

As mentioned earlier, behaviors such as substance use and events such as victimization 
are best measured by asking the individuals to repoit on discrete occurrences — thus, student 
surveys provide the best approach. To avoid the expensive proliferation of State-level 
student surveys, the Resource Group recommends using one study that is now being put in 
place at the State level, the Youth Risk Behavior Surveillance system, to measure these two 
elements. Use of YRBSS would require modest expansion of the content coverage of the 
questionnaire — but not an expansion that is incompatible with the current instrument (which 
deals with substance use, trauma, suicide, sexual behavior, and nutrition) nor one that should 
prove detrimental to the original purposes of the study. In fact, such an expansion to provide 
measurement for Goal 6 would very likely increase the States' motivation to cooperate with 
the survey and to implement it with greater methodological rigor. The specific suggestions 
regarding additions to this survey are given below, but it is worth mentioning here that the 
Resource Group sees a need for a number of States to improve the way in which their 
samples are drawn and their data are collected, if this system is to yield meaningful State- 
level data for Goal 6. (CDC does make sampling software and a procedures manual available 
to the States.) 
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A. Drug-Free Students and Schools 

1. National Indicators 

a. Recommended indicators and data sources. For this goal some of the same 
measures and data sources recommended for the 1991 Progress Report are also 
recommended for retention longer term, but some new indicators are proposed as well. 

(1) Student use of drugs. Continuation of the Monitoring the Future measures 
of use and grade of initiation of use (to provide lagged measures of changes at 
lower grades) is recommended. If new classes of drugs comr on the scene, these 
drugs will be added to the measurement package as has been done in the past. 

(2) Peer norms. Again, continuation of the MtF measures should fill this need at 
the national level. The extent of friends' disapproval is gathered for different 
levels of involvement (e.g., experimentation, occasional use, and regular use) for a 
number of drugs, including alcohol and cigarettes. 

(3) Being under the influence at school. This indicator is not currently a part of 
any ongoing national data collection system, which is based on self-report. 
Therefore, adding three questions to the MtF surveys is recommended: frequency 
of being under the influence of alcohol while at school, frequency of being under 
the influence of marijuana or any other illicit drug while at school, and frequency 
of smoking while at school. 

With regard f o teacher behavior, the approach of using staff respondents as 
observers of such behaviors among the staff generally could be adopted and the 
appropriate questions included in the Department of Education's Schools and 
Staffing Survey (SASS). This approach would permit biennial measurement 
beginning in 1991. Some concern was expressed with the rather pliable nature of 
the response scale to be used in that survey scries, but presumably a different one 
could be introduced for these three questions. 

(4) Sale or distribution at school. Sale and distribution is an illegal activity that 
can and does occur in the school's domain. There is at present no measurement of 
these behaviors at the national level. We recommend that a few questions be 
added to MtF about student awareness of drug dealing or distribution occurring in 
or near school. Direct self-reports are not suggested primarily for practical 
considerations: such questions are likely to arouse the suspicion of student 
respondents to the point where they become uncooperative with the survey in 
general. 
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2. State-Level Indicators 

It appears that enough of these indicators could be added to the CDCs Youth Risk 
Behavior Surveillance System to provide an adequate, though more limited, 
measurement package than at the national level Some States may choose to continue 
to rely on their own periodic statewide surveys of drug use for measuring progress 
against this element of Goal 6. Those that do may wish to consider adding some of the 
measures on "victimization in school" and "disciplined environment" to cover progress 
on the other two elements of Goal 6 as well — particularly if they are not expecting to 
implement the YRBSS on representative samples in their own State. 

a. Recommended indicators 

(1) Student use of drugs. The drug use measures already in YRBSS could serve 
rather well for State-level assessment. Serious consideration should be given to 
making the measurements comparable with the MtF survey on an item-by-item 
basis in the absence of compelling reasons to the contrary, at least on the key 
measures. States with their own series of student surveys on drug use may wish 
to make their measurements comparable with MtF for obvious reasons. A method 
exists for "splicing" trends from previous measures with those obtained with new 
measures by splitting the sample into random halves in one year and using the old 
questions with one half and the new ones with the other. 

(2) Peer norms. No State-level measures currently exist here, except perhaps in 
a few individual States with their own student survey series on dregs, but a very 
few questions from the larger set used in MtF might be added to YRBSS and/or 
State-specific surveys. 

(3) Being tinder the influence at school. The same three questions proposed 
above for addition to MtF for national monitoring purposes could be added to 
YRBSS and any State-specific surveys; i.e., questions on frequency of being 
under the influence of alcohol at school, being under the influence of marijuana or 
other illicit drugs at school, or smoking cigarettes at school. 

(4) Sale or distribution at school. Again, the few items to be added to MtF 
could be considered for inclusion in YRBSS and in State-specific surveys. 
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B. Schools Free of Violence and Crime. 

The indicators being recommended longer term for this element of Goal 6 are much the same 
as those proffered for the 1991 Progress Report, the main difference being that sowt of them 
are now recommended for inclusion in ongoing data collection systems. 

L National Indicators 

a. Recommended indicators and data sources 

(1) Victimization in school 

Students. The battery of seven student victimization items and one school 
property vandalism item in MtF should be retained and perhaps expanded if any 
additional victimization experiences are judged important for inclusion. Data on 
these items will then be available annually, beginning in late 1991, for 8th, 10th, 
and 12th grade students in public and private secondary schools nationwide. 
Teachers . The Schools and Staffing Survey could adopt the teacher victimization 
items included in the special 1991 Fast Response School Survey on violence in 
the schools, because the latter system is not intended to provide monitoring on n 
ongoing basis. These items would then be available on a national and State bi. ,s 
every other year, starting in 1993. 

(2) Feeling safe at school. 

Students. The 1989 School Crime Supplement to the National Crime Survey 
provided the measures of students' feelings of safety at school proposed for 
inclusion in the 1991 Progress Report. However, it is not clear whether, or how 
often, this supplement might be repeated. Further, there was some concern about 
the validity of such information gathered in the household setting. The most 
central questions used in the SCS could be included in the MtF series, which 
would provide annual measurement at the national level. 
Xcacheis. The questions in FRSS on teachers' feelings of safety in or near the 
school are excellent. Because FRSS is not intended for monitoring, some or all of 
these questions should be considered for inclusion in the Schools and Staffing 
Survey, which would provide both national and State teacher data on a biennial 
basis. 

(3) Carrying weapons to school. The 1989 School Crime Supplement, which is 
the preferred data source for the 1991 Progress Report, contained two excellent 
questions on this subject. Again, assuming that SCS will not be repeated, or in 
any case not often enough, one or both of the questions could be adapted to the 
MfF series for assessment on an annual basis. It has also been argued that 
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reporting on such deviant behavior is more likely to be valid in a school survey 
setting than in a household one. 

b. Considered but not recommended. Once again we have chosen to avoid the use of 
system-level records for the reasons given earlier. We have also avoided 
recommending the launching of new data-gathering series and have advised against 
reliance on one-shot or infrequently conducted studies, such as the School Crime 
Supplement and the FRSS. 

c. Other issues. The opposite side of the coin from victimization is the perpetration 
of crime. While reducing the criminal behavior of students is not one of the goals or 
objectives laid out by the Goals Panel, it may be one they would wish to consider. 
MtF already gathers data each year on a substantial number of delinquent acts (not 
specific to school), and the addition of others might be suggested based on Elliot's 
National Youth Survey, which is a panel study of delinquency and substance use among 
a set of adjacent birth cohorts passing through adolescence. 

2. State-Level Indicators 

The two State-level data systems most likely to implement measurement in this area 
are the Youth Risk Behavior Surveillance System of students and the School and 
Staffing Survey of teachers. 

a. Recommended indicators and data sources 



(1) Victimization at school. 

Students . YRBSS could adopt the seven-item scale of victimization in school 
now used in MtF, thus creating integrated national and State data. 
Ifcachfiis. SASS could adopt questions used on the 1991 FRSS survey of teachers, 
as has already been proposed for the national level. 

(2) Feeling safe at school. 

Students . Just as MtF could adopt selected questions in this domain for 
measurement at the national level, so could YRBSS add them to their State-»evel 
surveys (or at least provide them to the States as an optional component). 
Comparable data would then be available at the national and State levels. 
Teachers . Questions for FRSS have already been recommended for the teacher 
questionnaire of the national biennia! \SS surveys. These are conducted at the 
State level, ar well. 
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' (3) Carrying weapons at school. Again, the same one or two questions drawn 
from the School Crime Supplement to be added to MtF could be added to YRBSS 
(or offered to States as optional components), providing comparable State-level 
data in most States. 



C. Disciplined Environment Conducive To Learning 

Because a question remains regarding how broadly to define this element of Goal 6, the 
Resource Group needs some instruction from the Goals Panel. The issues are laid out in 
Section HI C in the discussion of the 1991 recommendations. For the moment, the Resource 
Group will take a fairly narrow interpretation of the term "disciplined environment conducive 
to learning" and recommend measurement of student misconduct in school and student and 
teacher perceptions of noise and disruption impeding learning. 

/. National Indicators: Recommended indicators and data sources 

(1) Student misconduct. Measures of student behaviors such as not cooperating 
with teachers, not following instructions in class, being disruptive in class, 
cheating on tests, copying others' homework, having verbal and physical 
altercations with other students in class, and having verbal and physical 
altercations with teachers in class should be included as measured variables. 
Perhaps the best approach for some of them is to ask how often these incidents 
happen in the student's classroom, while a few of the less visible behaviors (such 
as cheating and copying homework) may be best approached in self-reports of the 
respondent's behavior. °*V 

The few questions in this domain mentioned for the 1991 Progress Report are 
located in Monitoring the Future and NELS:88. Because the NELS:88 is not an 
ongoing monitoring system, it is recommended that a set of questions in this 
domain be selected and/or developed and inserted into the MtF survey. 

(2) Student and teacher perceptions of noise impeding learning. As was 
discussed in Section III C, the Resource Group considers measures based on 
frequency of occurrences to be the most appropriate for measuring change over 
time, as well as differences among States, because the standards for other Likert- 
type scales (e.g., "how big a problem is it, H "to what extent is it a problem") can 
vary between populations and across time. The data sources available for the 
1991 Progress Report were the 1990 NELS:88 and 1991 Fast Response Survey 
System, neither of which uses a frequency of occurrence scale on the few 
measures they do have of perceptions of noise and disruption impeding learning. 
Further, NELS:88 is not an ongoing monitoring system. 
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Thus, the Resource Group recommends new measures and new placements of 
them. Specifically, a very few questions (with parallel wording for students and 
teachers) could be added to the MtF series and the SASS series (and/or repeats of 
the 1991 FRSS survey). These questions would ask how frequently the classroom 
situation becomes so noisy and disruptive that it interferes with learning and 
teaching. The results on these measures could be expected to correlate strongly 
with the direct measures of student misconduct just discussed. 

(3) Other indicators. Should the Goals Panel choose to broaden the definition 
of a "disciplined environment conducive to learning" to include measures of 
tardiness for school and/or class, skipping school and/or class, hours spent on 
homework, and completion of homework, some measures currently in MtF could 
be used. Measures on homework completion and tardiness would need to be 
added. Because MtF has included a number of these measures for 15 years in the 
surveys of high school seniors, changes during the 1990s can be put into a longer- 
term historical perspective. 

« 

2. State-Level Indicators: Recommended data sources and measures 

Once again, use of the CDCs Youth Risk Behavior Surveillance System is 
recommended for gathering State-level student data on at least some of the same 
measures that are recommended for addition to Monitoring the Future surveys. 
(YRBSS may or may not choose to add these to its national high school survey 
component but could at least offer them as an optional component in the State-level 
measurement package.) The teacher questionnaire in the Schools and Staffing Survey 
is recommended for gathering the comparable teacher data at the State level. The 
measures would be the same as those gathered at the national level. 



V. Additional Recommendations 



A. Backup Technical Report 

The Resource Group recommends the creation of a more detailed technical report to 
supplement the necessarily oversimplified Progress Report. Under Goal 6, for instance, it 
might contain item-level trends for use of the different drugs, various types of victimization, 
various types of disorderly behavior in school, etc. The report also might contain data on the 
risk factor and precursor measures that reflect variables not included in the annual Progress 
Report, but which deal with conditions that must be changed in order to influence drug use, 
crime and violence, and orderliness in the school environment constructive to learning, for 
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example, student bonding to school and commitment to learning, community mobilization, 
etc. 



B. Super Objectives That Crosscut the Six Goals 

The Goals Panel may wish consider adding some measurable objectives that would be 
relevant to attaining Goal 6, but that also would be relevant to attaining most of the other 
goals, as well. These objectives would include such factors as bonding to school, feeling 
successful in school, being committed to learning, low truancy and tardiness rates, developing 
norms against deviant behavior, etc. They might also include parent involvement in the 
children's education and school, and the development of community support for and 
cooperation with the activities of the school. 



C. Local Data Collection 

Understanding that the President and Governors set these goals and will issue these reports on 
the State and national levels, the Resource Group was also mindful that success or failure in 
Goal 6 would largely depend on local efforts. Local districts and schools also need good data 
to assess their needs and gauge their own progress. The Resource Group therefore 
recommends that appropriate instruments and procedures be made available to, and their use 
be encouraged by, interested districts and agencies. For example, the Michigan Department 
of Education has developed a comprehensive package for a student survey compatible with 
the MtF measures, and a school prevention program assessment inventory. The package 
incorporates the use of standard sampling techniques, data collection procedures, and 
reporting methods through the use of an outside contractor whose services are available at 
low cost to school districts throughout the State. While the content emphasis is on substance 
abuse, the other elements of Goal 6— crime and a disciplined environment— could readily be 
woven into that effort or others like it. 
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PUBUC TESTIMONY 

Instructions: How to Submit Written Testimony for Pine! Consideration 

The National Education Goals Panel (NEGP) would like to receive testimony from all 
individuals and organizations wishing to comment on the selection of indicators, data 
sources, and measurement strategies relating to the national education goals for the 
September 1991 and future annual NEGP Progress Reports. 

To submit testimony for the Panel's consideration please till out the form on the 
opposite side of this page. You may photocopy the form and use both sides of the 
page, submitting a separate double-sided page for each goal upon which you want to 
comment. You may also submit one double-sided page to make general comments. 

You may attach additional written material, but only information submitted on the 
NEGP Public Testimony Form by May 12 will he summarized and reported to the 
Panel- This summary and highlights of individual testimony will be given to the full 
Panel before they make their decisions regarding the selection of indicators and data 
for the September 1991 Progress Report and future reports. Testimony submitted after 
May 12 cannot be considered regarding initial decisions about potential indicators and 
measurement strategies for monitoring progress toward the National Education Goals. 

Using the Written Public Testimony Form, please identify the individual and, if 
applicable, the exact organization or affiliate submitting testimony with the appropriate 
address, telephone number and Fax number. Also indicate the specific goal addressed 
by your comments. 
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NATIONAL EDUCATION GOALS PANEL 
Written Public Testimony Form 

To have your comments considered by the National Education Goals Panel, please complete 
this form. You may photocopy it and write front and back, submitting one two-sided page 
for each goal. You also may submit general comments. Please submit this form by May 12 
to: 

National Education Goals Panel 
Written Public Testimony 
1850 M Street, NW, Suite 270 
Washington, D.C. 20036 

FROM: 

Name: 

Organization (if any): 

Address: 



Telephone FAX 

SUBJECT: (Check appropriate space) 

Goal 1_2_3_4_5_6_ All Other 

Back-up Paper submitted Title 

Topic: 

Regional Forum (place, date, if appropriate) 
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